[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:35 dist-test-slave-bqcl krb5kdc[12361](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:35 dist-test-slave-bqcl krb5kdc[12361](info): set up 2 sockets
Aug 06 03:51:35 dist-test-slave-bqcl krb5kdc[12361](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260806 03:51:36.471496 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:36 dist-test-slave-bqcl krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:36.585124 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38491
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:39291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:38491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:36.670975 12383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:36.671183 12383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:36.671206 12383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:36.672335 12383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:36.672376 12383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:36.672390 12383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:36.672401 12383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:36.672412 12383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:36.674098 12383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:39291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:38491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:36.674496 12383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:36.674754 12383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:36.680545 12390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:36.680842 12383 server_base.cc:1034] running on GCE node
W20260806 03:51:36.682086 12391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:36.682353 12395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:36.682665 12383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:36.683132 12383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:36.684303 12383 hybrid_clock.cc:648] HybridClock initialized: now 1785988296684303 us; error 71 us; skew 500 ppm
Aug 06 03:51:36 dist-test-slave-bqcl krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988296, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:36.686916 12383 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:36.687579 12383 webserver.cc:460] Webserver started at http://127.12.15.126:36339/ using document root <none> and password file <none>
I20260806 03:51:36.687850 12383 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:36.687906 12383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:36.688066 12383 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:36.689170 12383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "5ad98fa7acb24ece88ff061dacc86af6"
format_stamp: "Formatted at 2026-08-06 03:51:36 on dist-test-slave-bqcl"
server_key: "d00766f24a0fe3d12dc620ac793323e6"
server_key_iv: "cebe6be52c3e07a01736f846eead6c7f"
server_key_version: "encryptionkey@0"
I20260806 03:51:36.689471 12383 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "5ad98fa7acb24ece88ff061dacc86af6"
format_stamp: "Formatted at 2026-08-06 03:51:36 on dist-test-slave-bqcl"
server_key: "d00766f24a0fe3d12dc620ac793323e6"
server_key_iv: "cebe6be52c3e07a01736f846eead6c7f"
server_key_version: "encryptionkey@0"
I20260806 03:51:36.691383 12383 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 03:51:36.692389 12402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:36.692654 12383 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:36.692740 12383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "5ad98fa7acb24ece88ff061dacc86af6"
format_stamp: "Formatted at 2026-08-06 03:51:36 on dist-test-slave-bqcl"
server_key: "d00766f24a0fe3d12dc620ac793323e6"
server_key_iv: "cebe6be52c3e07a01736f846eead6c7f"
server_key_version: "encryptionkey@0"
I20260806 03:51:36.692816 12383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:36.711386 12383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:36.713215 12383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:36.713380 12383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:36.720314 12383 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:38491
I20260806 03:51:36.720388 12481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:38491 every 8 connection(s)
I20260806 03:51:36.721015 12383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:36.722605 12482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:36.726670 12482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6: Bootstrap starting.
I20260806 03:51:36.728121 12482 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:36.728905 12482 log.cc:824] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:36.729867 12482 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6: No bootstrap required, opened a new log
I20260806 03:51:36.730814 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12383
I20260806 03:51:36.730898 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:36.731484 12349 external_mini_cluster.cc:1428] Setting key fa2d4cd86025c9fb07ec0a86531909cc
I20260806 03:51:36.733387 12482 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad98fa7acb24ece88ff061dacc86af6" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38491 } }
I20260806 03:51:36.733556 12482 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:36.733691 12482 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ad98fa7acb24ece88ff061dacc86af6, State: Initialized, Role: FOLLOWER
I20260806 03:51:36.733855 12482 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad98fa7acb24ece88ff061dacc86af6" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38491 } }
I20260806 03:51:36.734014 12482 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:36.734079 12482 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:36.734169 12482 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:36.734822 12482 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad98fa7acb24ece88ff061dacc86af6" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38491 } }
I20260806 03:51:36.735025 12482 leader_election.cc:302] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ad98fa7acb24ece88ff061dacc86af6; no voters:
I20260806 03:51:36.735251 12482 leader_election.cc:288] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:36.735284 12489 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:36.735646 12489 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [term 1 LEADER]: Becoming Leader. State: Replica: 5ad98fa7acb24ece88ff061dacc86af6, State: Running, Role: LEADER
I20260806 03:51:36.735839 12489 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad98fa7acb24ece88ff061dacc86af6" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38491 } }
I20260806 03:51:36.735991 12482 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:36.736613 12490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ad98fa7acb24ece88ff061dacc86af6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad98fa7acb24ece88ff061dacc86af6" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38491 } } }
I20260806 03:51:36.736639 12491 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ad98fa7acb24ece88ff061dacc86af6. Latest consensus state: current_term: 1 leader_uuid: "5ad98fa7acb24ece88ff061dacc86af6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ad98fa7acb24ece88ff061dacc86af6" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38491 } } }
I20260806 03:51:36.736704 12490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:36.736725 12491 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:36.737473 12498 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:36.739472 12498 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:36.741710 12498 catalog_manager.cc:1347] Generated new cluster ID: abc44223d9fd41c68f89ebf587856dce
I20260806 03:51:36.741767 12498 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:51:36 dist-test-slave-bqcl krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988296, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:36.753764 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:36.735383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40174 (local address 127.12.15.126:38491)
0806 03:51:36.735969 (+ 586us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:36.735979 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:36.736037 (+ 58us) server_negotiation.cc:407] Connection header received
0806 03:51:36.736220 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:36.736225 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:36.736306 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:36.736426 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:36.739009 (+ 2583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:36.741018 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:36.742607 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:36.743200 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:36.743295 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:36.748056 (+ 4761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:36.748061 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:36.748061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:36.748072 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:36.751374 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:36.751878 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:36.751881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:36.751881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:36.752041 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:36.752361 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:36.752362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:36.752363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:36.752733 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:36.753028 (+ 295us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:36.753268 (+ 240us) server_negotiation.cc:299] Negotiation successful
0806 03:51:36.753445 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":336,"threads_started":1}
I20260806 03:51:36.760494 12498 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:36.761274 12498 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:36.767087 12498 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ad98fa7acb24ece88ff061dacc86af6: Generated new TSK 0
I20260806 03:51:36.767261 12498 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:51:36.797827 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38491
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:39291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:36.915089 12521 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:36.915305 12521 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:36.915340 12521 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:36.916531 12521 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:36.916579 12521 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:36.916617 12521 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:36.918051 12521 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:39291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38491
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12521
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:36.918404 12521 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:36.918642 12521 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:36.924640 12533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:36.924970 12530 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:36.925040 12531 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:36.925989 12521 server_base.cc:1034] running on GCE node
I20260806 03:51:36.926232 12521 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:36.926468 12521 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:36.927624 12521 hybrid_clock.cc:648] HybridClock initialized: now 1785988296927602 us; error 36 us; skew 500 ppm
Aug 06 03:51:36 dist-test-slave-bqcl krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988296, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:36.930516 12521 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:51:36.930977 12521 webserver.cc:460] Webserver started at http://127.12.15.65:45521/ using document root <none> and password file <none>
I20260806 03:51:36.931191 12521 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:36.931244 12521 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:36.931377 12521 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:36.932511 12521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "3888cd12b36740ee82731f090f5882e9"
format_stamp: "Formatted at 2026-08-06 03:51:36 on dist-test-slave-bqcl"
server_key: "3f4c6b0236968c131465c40c2322ee61"
server_key_iv: "b7260662322fe405a39eb1a4a05466d3"
server_key_version: "encryptionkey@0"
I20260806 03:51:36.932926 12521 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "3888cd12b36740ee82731f090f5882e9"
format_stamp: "Formatted at 2026-08-06 03:51:36 on dist-test-slave-bqcl"
server_key: "3f4c6b0236968c131465c40c2322ee61"
server_key_iv: "b7260662322fe405a39eb1a4a05466d3"
server_key_version: "encryptionkey@0"
I20260806 03:51:36.934281 12521 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 03:51:36.935053 12546 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:36.935232 12521 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:51:36.935323 12521 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "3888cd12b36740ee82731f090f5882e9"
format_stamp: "Formatted at 2026-08-06 03:51:36 on dist-test-slave-bqcl"
server_key: "3f4c6b0236968c131465c40c2322ee61"
server_key_iv: "b7260662322fe405a39eb1a4a05466d3"
server_key_version: "encryptionkey@0"
I20260806 03:51:36.935389 12521 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:36.946798 12521 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:36.948772 12521 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:36.948954 12521 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:36.949359 12521 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:36.949761 12521 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:36.949854 12521 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:36.949893 12521 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:36.949921 12521 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:36.965898 12521 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:33381
I20260806 03:51:36.965955 12703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:33381 every 8 connection(s)
I20260806 03:51:36.966302 12521 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:51:36 dist-test-slave-bqcl krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988296, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:36.974016 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12521
I20260806 03:51:36.974097 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:36.974242 12349 external_mini_cluster.cc:1428] Setting key 156641281cbca6393e4fee260908c44b
I20260806 03:51:36.976579 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:36.967211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:41133 (local address 127.12.15.126:38491)
0806 03:51:36.967260 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:36.967262 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:36.967553 (+ 291us) server_negotiation.cc:407] Connection header received
0806 03:51:36.968353 (+ 800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:36.968354 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:36.968369 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:36.968416 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:36.969573 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:36.970068 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:36.971225 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:36.971429 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:36.971492 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:36.973835 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:36.973836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:36.973837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:36.973845 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:36.975176 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:36.975601 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:36.975602 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:36.975602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:36.975705 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:36.975947 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:36.975947 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:36.975947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:36.976046 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:36.976091 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:36.976387 (+ 296us) server_negotiation.cc:299] Negotiation successful
0806 03:51:36.976476 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:51:36.976747 12707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:36.967181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38491 (local address 127.12.15.65:41133)
0806 03:51:36.967404 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:36.967441 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:36.968247 (+ 806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:36.968496 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:36.968499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:36.969161 (+ 662us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:36.969477 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:36.969482 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:36.970168 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:36.970168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:36.971141 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:36.971143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:36.971497 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:36.971499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:36.971667 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:36.972220 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:36.972229 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:36.973759 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:36.975248 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:36.975256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:36.975258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:36.975532 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:36.975761 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:36.975762 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:36.975763 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:36.975893 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:36.976094 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:36.976099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:36.976255 (+ 156us) client_negotiation.cc:769] Sending connection context
0806 03:51:36.976357 (+ 102us) client_negotiation.cc:240] Negotiation successful
0806 03:51:36.976526 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260806 03:51:36.977378 12704 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38491
I20260806 03:51:36.977461 12704 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:36.977626 12704 heartbeater.cc:511] Master 127.12.15.126:38491 requested a full tablet report, sending...
I20260806 03:51:36.978441 12430 ts_manager.cc:194] Registered new tserver with Master: 3888cd12b36740ee82731f090f5882e9 (127.12.15.65:33381)
I20260806 03:51:36.979177 12430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:41133
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:51:37.002285 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38491
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:39291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:37.087467 12710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:37.087688 12710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:37.087713 12710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:37.088788 12710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:37.088824 12710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:37.088850 12710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:37.090255 12710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:39291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38491
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:37.090438 12710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:37.090632 12710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:37.096350 12717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:37.096385 12719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:37.096422 12716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:37.096463 12710 server_base.cc:1034] running on GCE node
I20260806 03:51:37.096733 12710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:37.096971 12710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:37.098101 12710 hybrid_clock.cc:648] HybridClock initialized: now 1785988297098085 us; error 34 us; skew 500 ppm
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988297, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:37.100466 12710 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:51:37.100862 12710 webserver.cc:460] Webserver started at http://127.12.15.66:33101/ using document root <none> and password file <none>
I20260806 03:51:37.101043 12710 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:37.101089 12710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:37.101220 12710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:37.102281 12710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "13e6889b7ec74162a3f6d1a4e08f9280"
format_stamp: "Formatted at 2026-08-06 03:51:37 on dist-test-slave-bqcl"
server_key: "d5f0f20f39343dca61ab698493746ff6"
server_key_iv: "bc347617ac64ee112b085a7ef1ade383"
server_key_version: "encryptionkey@0"
I20260806 03:51:37.102644 12710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "13e6889b7ec74162a3f6d1a4e08f9280"
format_stamp: "Formatted at 2026-08-06 03:51:37 on dist-test-slave-bqcl"
server_key: "d5f0f20f39343dca61ab698493746ff6"
server_key_iv: "bc347617ac64ee112b085a7ef1ade383"
server_key_version: "encryptionkey@0"
I20260806 03:51:37.103912 12710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:51:37.104557 12729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:37.104704 12710 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:37.104786 12710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "13e6889b7ec74162a3f6d1a4e08f9280"
format_stamp: "Formatted at 2026-08-06 03:51:37 on dist-test-slave-bqcl"
server_key: "d5f0f20f39343dca61ab698493746ff6"
server_key_iv: "bc347617ac64ee112b085a7ef1ade383"
server_key_version: "encryptionkey@0"
I20260806 03:51:37.104843 12710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:37.118728 12710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:37.120337 12710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:37.120465 12710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:37.120661 12710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:37.120945 12710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:37.120978 12710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:37.121007 12710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:37.121031 12710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:37.126571 12710 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:37463
I20260806 03:51:37.126631 12903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:37463 every 8 connection(s)
I20260806 03:51:37.126906 12710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:51:37.127352 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12710
I20260806 03:51:37.127434 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:37.127610 12349 external_mini_cluster.cc:1428] Setting key ffdad825131e17e04b8143aeb95e45dc
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988297, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:37.136053 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:37.127541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:45773 (local address 127.12.15.126:38491)
0806 03:51:37.127612 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:37.127614 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:37.127809 (+ 195us) server_negotiation.cc:407] Connection header received
0806 03:51:37.128376 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:37.128377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:37.128386 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:37.128419 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:37.129559 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.130056 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.131037 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.131243 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.131300 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:37.133412 (+ 2112us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:37.133413 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:37.133413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:37.133420 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:37.134697 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:37.135146 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:37.135147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:37.135147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:37.135201 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:37.135447 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:37.135447 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:37.135447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:37.135536 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:37.135578 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:37.135918 (+ 340us) server_negotiation.cc:299] Negotiation successful
0806 03:51:37.135989 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:51:37.136301 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:37.127501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38491 (local address 127.12.15.66:45773)
0806 03:51:37.127680 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:37.127729 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:37.128281 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:37.128498 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:37.128502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:37.129223 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:37.129481 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:37.129488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.130161 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.130162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:37.130957 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:37.130960 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.131315 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.131316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:37.131442 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:37.131874 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:37.131882 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:37.133321 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:37.134774 (+ 1453us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:37.134781 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:37.134783 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:37.135076 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:37.135255 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:37.135256 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:37.135257 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:37.135390 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:37.135591 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:37.135597 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:37.135782 (+ 185us) client_negotiation.cc:769] Sending connection context
0806 03:51:37.135902 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 03:51:37.136064 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":84,"threads_started":1}
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
I20260806 03:51:37.136853 12904 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38491
I20260806 03:51:37.136953 12904 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:37.137135 12904 heartbeater.cc:511] Master 127.12.15.126:38491 requested a full tablet report, sending...
I20260806 03:51:37.137496 12430 ts_manager.cc:194] Registered new tserver with Master: 13e6889b7ec74162a3f6d1a4e08f9280 (127.12.15.66:37463)
I20260806 03:51:37.137881 12430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:45773
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:51:37.153196 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38491
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:39291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:37.234689 12914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:37.234884 12914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:37.234905 12914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:37.236049 12914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:37.236092 12914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:37.236117 12914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:37.237641 12914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:39291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38491
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:37.237854 12914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:37.238059 12914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:37.242851 12923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:37.242861 12924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:37.242872 12927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:37.243130 12914 server_base.cc:1034] running on GCE node
I20260806 03:51:37.243333 12914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:37.243566 12914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:37.244717 12914 hybrid_clock.cc:648] HybridClock initialized: now 1785988297244704 us; error 28 us; skew 500 ppm
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988297, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:37.247176 12914 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:51:37.247605 12914 webserver.cc:460] Webserver started at http://127.12.15.67:36069/ using document root <none> and password file <none>
I20260806 03:51:37.247830 12914 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:37.247880 12914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:37.248008 12914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:37.249015 12914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "d6822df953a546c4b3607a6f37b1108d"
format_stamp: "Formatted at 2026-08-06 03:51:37 on dist-test-slave-bqcl"
server_key: "d5e823e499cfc971feae37d2d6129651"
server_key_iv: "c2ca8a132934e84a593949fb07d8ef02"
server_key_version: "encryptionkey@0"
I20260806 03:51:37.249408 12914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "d6822df953a546c4b3607a6f37b1108d"
format_stamp: "Formatted at 2026-08-06 03:51:37 on dist-test-slave-bqcl"
server_key: "d5e823e499cfc971feae37d2d6129651"
server_key_iv: "c2ca8a132934e84a593949fb07d8ef02"
server_key_version: "encryptionkey@0"
I20260806 03:51:37.250809 12914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:37.251448 12937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:37.251593 12914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:37.251685 12914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "d6822df953a546c4b3607a6f37b1108d"
format_stamp: "Formatted at 2026-08-06 03:51:37 on dist-test-slave-bqcl"
server_key: "d5e823e499cfc971feae37d2d6129651"
server_key_iv: "c2ca8a132934e84a593949fb07d8ef02"
server_key_version: "encryptionkey@0"
I20260806 03:51:37.251745 12914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:37.262511 12914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:37.264526 12914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:37.264670 12914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:37.264873 12914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:37.265146 12914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:37.265182 12914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:37.265209 12914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:37.265244 12914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:37.271273 12914 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:34531
I20260806 03:51:37.271330 13146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:34531 every 8 connection(s)
I20260806 03:51:37.271617 12914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988297, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:37.277982 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12914
I20260806 03:51:37.278057 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:37.278234 12349 external_mini_cluster.cc:1428] Setting key ffc209ceb3e5e35bd4841df8fc38bc7b
I20260806 03:51:37.280021 12492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:37.272305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:48277 (local address 127.12.15.126:38491)
0806 03:51:37.272381 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:37.272384 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:37.272577 (+ 193us) server_negotiation.cc:407] Connection header received
0806 03:51:37.273037 (+ 460us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:37.273038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:37.273048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:37.273096 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:37.273932 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.274467 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.275408 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.275584 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.275624 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:37.277475 (+ 1851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:37.277477 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:37.277477 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:37.277483 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:37.278820 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:37.279247 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:37.279249 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:37.279249 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:37.279316 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:37.279541 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:37.279542 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:37.279542 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:37.279650 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:37.279713 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:37.279900 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 03:51:37.279959 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 03:51:37.280171 13151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:37.272284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38491 (local address 127.12.15.67:48277)
0806 03:51:37.272464 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:37.272499 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:37.272961 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:37.273149 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:37.273152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:37.273672 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:37.273868 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:37.273873 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.274538 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.274538 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:37.275346 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:37.275347 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.275631 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.275631 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:37.275734 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:37.276102 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:37.276109 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:37.277395 (+ 1286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:37.278884 (+ 1489us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:37.278890 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:37.278892 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:37.279169 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:37.279373 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:37.279374 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:37.279374 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:37.279487 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:37.279709 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:37.279712 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:37.279812 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 03:51:37.279886 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:51:37.279995 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260806 03:51:37.280551 13148 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38491
I20260806 03:51:37.280625 13148 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:37.280763 13148 heartbeater.cc:511] Master 127.12.15.126:38491 requested a full tablet report, sending...
I20260806 03:51:37.281037 12430 ts_manager.cc:194] Registered new tserver with Master: d6822df953a546c4b3607a6f37b1108d (127.12.15.67:34531)
I20260806 03:51:37.281378 12430 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:48277
I20260806 03:51:37.290076 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988297, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988297, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:51:37.313297 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:37.306181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35902 (local address 127.12.15.65:33381)
0806 03:51:37.306364 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:37.306369 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:37.306380 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:51:37.306406 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:37.306408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:37.306423 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:37.306492 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:37.307003 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.307786 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.308687 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.309010 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.309064 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:37.311082 (+ 2018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:37.311090 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:37.311091 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:37.311097 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:37.312378 (+ 1281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:37.312692 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:37.312695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:37.312695 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:37.312743 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:37.312874 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:37.312875 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:37.312875 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:37.312987 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:37.313120 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:37.313152 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 03:51:37.313220 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":97,"threads_started":1}
W20260806 03:51:37.313746 12585 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35902
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[12361](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[12361](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988297, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:51:37.335196 13159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:37.329056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.12.15.65:33381)
0806 03:51:37.329125 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:37.329127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:37.329184 (+ 57us) server_negotiation.cc:407] Connection header received
0806 03:51:37.329240 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:37.329241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:37.329252 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:37.329298 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:37.329776 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.330298 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.330969 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:37.331156 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:37.331209 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:37.333122 (+ 1913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:37.333123 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:37.333124 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:37.333131 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:37.334368 (+ 1237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:37.334642 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:37.334642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:37.334643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:37.334696 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:37.334829 (+ 133us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:37.334830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:37.334830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:37.334916 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:37.335013 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:37.335062 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 03:51:37.335129 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 03:51:37.335749 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12521
I20260806 03:51:37.351768 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12710
I20260806 03:51:37.359051 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12914
I20260806 03:51:37.368924 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12383
2026-08-06T03:51:37Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2049 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[13169](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[13169](info): set up 2 sockets
Aug 06 03:51:37 dist-test-slave-bqcl krb5kdc[13169](info): commencing operation
krb5kdc: starting...
W20260806 03:51:38.503723 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:38 dist-test-slave-bqcl krb5kdc[13169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:38.614605 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:43497
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:43497
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:38.694803 13190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:38.695019 13190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:38.695039 13190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:38.696148 13190 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:38.696200 13190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:38.696220 13190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:38.696237 13190 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:38.696260 13190 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:38.698369 13190 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:43497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:43497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:38.698638 13190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:38.698915 13190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:38.704550 13206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:38.704555 13200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:38.704641 13198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:38.705538 13190 server_base.cc:1034] running on GCE node
I20260806 03:51:38.705790 13190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:38.706058 13190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:38.707207 13190 hybrid_clock.cc:648] HybridClock initialized: now 1785988298707193 us; error 30 us; skew 500 ppm
Aug 06 03:51:38 dist-test-slave-bqcl krb5kdc[13169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988298, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:38.709538 13190 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:38.709959 13190 webserver.cc:460] Webserver started at http://127.12.15.126:33983/ using document root <none> and password file <none>
I20260806 03:51:38.710191 13190 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:38.710243 13190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:38.710367 13190 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:38.711382 13190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "8d2dbb728a05498da146fb2f30ab4fdb"
format_stamp: "Formatted at 2026-08-06 03:51:38 on dist-test-slave-bqcl"
server_key: "a278643cd2269792d7f1b26ef2f9a4e2"
server_key_iv: "4fdfd84c899fad0349dc7394eb8294ed"
server_key_version: "encryptionkey@0"
I20260806 03:51:38.711715 13190 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "8d2dbb728a05498da146fb2f30ab4fdb"
format_stamp: "Formatted at 2026-08-06 03:51:38 on dist-test-slave-bqcl"
server_key: "a278643cd2269792d7f1b26ef2f9a4e2"
server_key_iv: "4fdfd84c899fad0349dc7394eb8294ed"
server_key_version: "encryptionkey@0"
I20260806 03:51:38.712977 13190 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:38.713573 13214 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:38.713733 13190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:38.713812 13190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "8d2dbb728a05498da146fb2f30ab4fdb"
format_stamp: "Formatted at 2026-08-06 03:51:38 on dist-test-slave-bqcl"
server_key: "a278643cd2269792d7f1b26ef2f9a4e2"
server_key_iv: "4fdfd84c899fad0349dc7394eb8294ed"
server_key_version: "encryptionkey@0"
I20260806 03:51:38.713872 13190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:38.762159 13190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:38.763967 13190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:38.764101 13190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:38.767900 13190 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:43497
I20260806 03:51:38.767956 13292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:43497 every 8 connection(s)
I20260806 03:51:38.768273 13190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:38.769212 13293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:38.769464 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13190
I20260806 03:51:38.769536 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:38.769693 12349 external_mini_cluster.cc:1428] Setting key 88524e16f80cbdb8fddb9844d8d38ec8
I20260806 03:51:38.772380 13293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb: Bootstrap starting.
I20260806 03:51:38.773172 13293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:38.773468 13293 log.cc:824] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:38.774192 13293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb: No bootstrap required, opened a new log
Aug 06 03:51:38 dist-test-slave-bqcl krb5kdc[13169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:38.776686 13293 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d2dbb728a05498da146fb2f30ab4fdb" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43497 } }
I20260806 03:51:38.776855 13293 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:38.776907 13293 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8d2dbb728a05498da146fb2f30ab4fdb, State: Initialized, Role: FOLLOWER
I20260806 03:51:38.777045 13293 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d2dbb728a05498da146fb2f30ab4fdb" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43497 } }
I20260806 03:51:38.777132 13293 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:38.777161 13293 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:38.777204 13293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:38.777776 13293 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d2dbb728a05498da146fb2f30ab4fdb" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43497 } }
I20260806 03:51:38.777879 13293 leader_election.cc:302] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8d2dbb728a05498da146fb2f30ab4fdb; no voters:
I20260806 03:51:38.778038 13293 leader_election.cc:288] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:38.778081 13300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:38.778215 13300 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [term 1 LEADER]: Becoming Leader. State: Replica: 8d2dbb728a05498da146fb2f30ab4fdb, State: Running, Role: LEADER
I20260806 03:51:38.778295 13300 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d2dbb728a05498da146fb2f30ab4fdb" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43497 } }
I20260806 03:51:38.778316 13293 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:38.778789 13303 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8d2dbb728a05498da146fb2f30ab4fdb. Latest consensus state: current_term: 1 leader_uuid: "8d2dbb728a05498da146fb2f30ab4fdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d2dbb728a05498da146fb2f30ab4fdb" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43497 } } }
I20260806 03:51:38.778875 13303 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:38.779103 13309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:38.779666 13301 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8d2dbb728a05498da146fb2f30ab4fdb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8d2dbb728a05498da146fb2f30ab4fdb" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43497 } } }
I20260806 03:51:38.779740 13301 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:38.779794 13309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:38.780161 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:38.770204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.12.15.126:43497)
0806 03:51:38.770376 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:38.770384 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:38.770409 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:51:38.770832 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:38.770839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:38.770887 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:38.770997 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:38.771739 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:38.772621 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:38.773621 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:38.774024 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:38.774101 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:38.776507 (+ 2406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:38.776510 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:38.776511 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:38.776522 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:38.778168 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:38.778572 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:38.778575 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:38.778575 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:38.778678 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:38.778899 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:38.778900 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:38.778901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:38.779225 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:38.779372 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:38.779825 (+ 453us) server_negotiation.cc:299] Negotiation successful
0806 03:51:38.779960 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":84,"threads_started":1}
I20260806 03:51:38.781291 13309 catalog_manager.cc:1347] Generated new cluster ID: c5edf5ca98824751a2e2c9be708c5629
I20260806 03:51:38.781332 13309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:38.795957 13309 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:38.796413 13309 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:38.801156 13309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8d2dbb728a05498da146fb2f30ab4fdb: Generated new TSK 0
I20260806 03:51:38.801301 13309 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:51:38.828215 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43497
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:38.924949 13337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:38.925145 13337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:38.925168 13337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:38.926342 13337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:38.926383 13337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:38.926410 13337 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:38.927896 13337 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:38.928102 13337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:38.928318 13337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:38.933569 13347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:38.933621 13349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:38.933620 13354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:38.933743 13337 server_base.cc:1034] running on GCE node
I20260806 03:51:38.934022 13337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:38.934271 13337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:38.935417 13337 hybrid_clock.cc:648] HybridClock initialized: now 1785988298935403 us; error 32 us; skew 500 ppm
Aug 06 03:51:38 dist-test-slave-bqcl krb5kdc[13169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988298, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:38.937796 13337 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:51:38.938217 13337 webserver.cc:460] Webserver started at http://127.12.15.65:38061/ using document root <none> and password file <none>
I20260806 03:51:38.938426 13337 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:38.938472 13337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:38.938598 13337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:38.939548 13337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "aee30ce814a0478b8232f99906c9a84e"
format_stamp: "Formatted at 2026-08-06 03:51:38 on dist-test-slave-bqcl"
server_key: "e6d487b6f1a6520b09fc8d2ebc3b77c5"
server_key_iv: "36121ed1df81a9c09545322668404a9a"
server_key_version: "encryptionkey@0"
I20260806 03:51:38.939877 13337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "aee30ce814a0478b8232f99906c9a84e"
format_stamp: "Formatted at 2026-08-06 03:51:38 on dist-test-slave-bqcl"
server_key: "e6d487b6f1a6520b09fc8d2ebc3b77c5"
server_key_iv: "36121ed1df81a9c09545322668404a9a"
server_key_version: "encryptionkey@0"
I20260806 03:51:38.941169 13337 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:38.941870 13363 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:38.942018 13337 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:38.942090 13337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "aee30ce814a0478b8232f99906c9a84e"
format_stamp: "Formatted at 2026-08-06 03:51:38 on dist-test-slave-bqcl"
server_key: "e6d487b6f1a6520b09fc8d2ebc3b77c5"
server_key_iv: "36121ed1df81a9c09545322668404a9a"
server_key_version: "encryptionkey@0"
I20260806 03:51:38.942178 13337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:38.967095 13337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:38.968779 13337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:38.968909 13337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:38.969120 13337 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:38.969412 13337 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:38.969445 13337 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:38.969473 13337 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:38.969489 13337 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:38.974960 13337 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:39961
I20260806 03:51:38.975018 13534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:39961 every 8 connection(s)
I20260806 03:51:38.975301 13337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:51:38 dist-test-slave-bqcl krb5kdc[13169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988298, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:38.983961 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:38.975947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:39059 (local address 127.12.15.126:43497)
0806 03:51:38.975989 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:38.975991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:38.976176 (+ 185us) server_negotiation.cc:407] Connection header received
0806 03:51:38.977180 (+ 1004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:38.977181 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:38.977191 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:38.977241 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:38.977844 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:38.978380 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:38.979409 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:38.979620 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:38.979675 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:38.981625 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:38.981626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:38.981626 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:38.981634 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:38.982749 (+ 1115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:38.983111 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:38.983112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:38.983112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:38.983164 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:38.983371 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:38.983372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:38.983372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:38.983479 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:38.983535 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:38.983786 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 03:51:38.983871 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:51:38.984126 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:38.975908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:43497 (local address 127.12.15.65:39059)
0806 03:51:38.976062 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:38.976104 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:38.977080 (+ 976us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:38.977305 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:38.977308 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:38.977554 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:38.977760 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:38.977766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:38.978439 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:38.978440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:38.979345 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:38.979348 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:38.979690 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:38.979691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:38.979795 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:38.980204 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:38.980211 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:38.981529 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:38.982820 (+ 1291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:38.982827 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:38.982829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:38.983038 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:38.983218 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:38.983219 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:38.983219 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:38.983316 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:38.983534 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:38.983540 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:38.983672 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 03:51:38.983759 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:51:38.983910 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260806 03:51:38.984577 13535 heartbeater.cc:347] Connected to a master server at 127.12.15.126:43497
I20260806 03:51:38.984665 13535 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:38.984836 13535 heartbeater.cc:511] Master 127.12.15.126:43497 requested a full tablet report, sending...
I20260806 03:51:38.985018 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13337
I20260806 03:51:38.985124 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:38.985306 12349 external_mini_cluster.cc:1428] Setting key ccfead9cdb8c782123d6a70496115def
I20260806 03:51:38.985385 13240 ts_manager.cc:194] Registered new tserver with Master: aee30ce814a0478b8232f99906c9a84e (127.12.15.65:39961)
I20260806 03:51:38.986320 13240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:39059
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:51:39.009594 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43497
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:39.089474 13540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:39.089684 13540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:39.089706 13540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:39.090857 13540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:39.090898 13540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:39.090926 13540 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:39.092391 13540 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13540
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:39.092608 13540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:39.092831 13540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:51:39.097649 13540 server_base.cc:1034] running on GCE node
W20260806 03:51:39.097596 13546 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:39.097626 13547 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:39.097617 13550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:39.098069 13540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:39.098323 13540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:39.099470 13540 hybrid_clock.cc:648] HybridClock initialized: now 1785988299099455 us; error 30 us; skew 500 ppm
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988299, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:39.101819 13540 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:51:39.102360 13540 webserver.cc:460] Webserver started at http://127.12.15.66:41819/ using document root <none> and password file <none>
I20260806 03:51:39.102582 13540 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:39.102631 13540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:39.102757 13540 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:39.103832 13540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "de422f064a964722b2b79f6eff551438"
format_stamp: "Formatted at 2026-08-06 03:51:39 on dist-test-slave-bqcl"
server_key: "befcfaf7efd060e20026df0c71f45104"
server_key_iv: "4459ac3be5cd4506d6acadb55ad78cc4"
server_key_version: "encryptionkey@0"
I20260806 03:51:39.104157 13540 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "de422f064a964722b2b79f6eff551438"
format_stamp: "Formatted at 2026-08-06 03:51:39 on dist-test-slave-bqcl"
server_key: "befcfaf7efd060e20026df0c71f45104"
server_key_iv: "4459ac3be5cd4506d6acadb55ad78cc4"
server_key_version: "encryptionkey@0"
I20260806 03:51:39.105381 13540 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:39.106010 13559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:39.106191 13540 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:39.106277 13540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "de422f064a964722b2b79f6eff551438"
format_stamp: "Formatted at 2026-08-06 03:51:39 on dist-test-slave-bqcl"
server_key: "befcfaf7efd060e20026df0c71f45104"
server_key_iv: "4459ac3be5cd4506d6acadb55ad78cc4"
server_key_version: "encryptionkey@0"
I20260806 03:51:39.106344 13540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:39.115044 13540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:39.116701 13540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:39.116837 13540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:39.117039 13540 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:39.117375 13540 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:39.117412 13540 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:39.117440 13540 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:39.117457 13540 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:39.123000 13540 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:33607
I20260806 03:51:39.123064 13731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:33607 every 8 connection(s)
I20260806 03:51:39.123349 13540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:51:39.123984 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13540
I20260806 03:51:39.124076 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:39.124248 12349 external_mini_cluster.cc:1428] Setting key 94d6d0ddc5fa4ac82a0cf5265bde7b2e
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988299, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:39.132858 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.124161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:58495 (local address 127.12.15.126:43497)
0806 03:51:39.124208 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.124210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.124365 (+ 155us) server_negotiation.cc:407] Connection header received
0806 03:51:39.125381 (+ 1016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.125382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.125394 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.125444 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:39.126054 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.126583 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.127557 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.127750 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.127806 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.130022 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:39.130023 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:39.130024 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:39.130032 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:39.131486 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.131868 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.131869 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.131869 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.131927 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.132136 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.132137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.132137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.132359 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:39.132436 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.132655 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.132758 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:51:39.132897 13736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.123964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:43497 (local address 127.12.15.66:58495)
0806 03:51:39.124248 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:39.124293 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:39.125282 (+ 989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:39.125518 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:39.125522 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:39.125783 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:39.125966 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.125970 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.126674 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.126675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.127474 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.127477 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.127819 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.127820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.127942 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.128394 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:39.128402 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:39.129922 (+ 1520us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:39.131563 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:39.131570 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:39.131572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:39.131797 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:39.131987 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:39.131988 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:39.131988 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:39.132082 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:39.132422 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:39.132425 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:39.132533 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 03:51:39.132607 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:51:39.132735 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":83,"threads_started":1}
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
I20260806 03:51:39.133330 13733 heartbeater.cc:347] Connected to a master server at 127.12.15.126:43497
I20260806 03:51:39.133430 13733 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:39.133594 13733 heartbeater.cc:511] Master 127.12.15.126:43497 requested a full tablet report, sending...
I20260806 03:51:39.133939 13240 ts_manager.cc:194] Registered new tserver with Master: de422f064a964722b2b79f6eff551438 (127.12.15.66:33607)
I20260806 03:51:39.134415 13240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:58495
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:51:39.149923 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43497
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:39.230590 13739 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:39.230775 13739 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:39.230798 13739 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:39.231889 13739 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:39.231927 13739 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:39.231956 13739 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:39.233397 13739 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43497
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13739
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:39.233608 13739 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:39.233814 13739 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:39.238485 13744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:39.238664 13739 server_base.cc:1034] running on GCE node
W20260806 03:51:39.238521 13748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:39.240190 13745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:39.240420 13739 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:39.240640 13739 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:39.241777 13739 hybrid_clock.cc:648] HybridClock initialized: now 1785988299241762 us; error 29 us; skew 500 ppm
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988299, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:39.244197 13739 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:51:39.244616 13739 webserver.cc:460] Webserver started at http://127.12.15.67:41629/ using document root <none> and password file <none>
I20260806 03:51:39.244841 13739 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:39.244894 13739 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:39.245014 13739 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:39.245999 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "944a801e66724c0cab5fe90582e1b100"
format_stamp: "Formatted at 2026-08-06 03:51:39 on dist-test-slave-bqcl"
server_key: "51fe63398728c2647b1051e9639ca6b5"
server_key_iv: "029170bc0d628721fdbbf5567bd6df0d"
server_key_version: "encryptionkey@0"
I20260806 03:51:39.246366 13739 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "944a801e66724c0cab5fe90582e1b100"
format_stamp: "Formatted at 2026-08-06 03:51:39 on dist-test-slave-bqcl"
server_key: "51fe63398728c2647b1051e9639ca6b5"
server_key_iv: "029170bc0d628721fdbbf5567bd6df0d"
server_key_version: "encryptionkey@0"
I20260806 03:51:39.247705 13739 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:39.248355 13758 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:39.248520 13739 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:39.248593 13739 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "944a801e66724c0cab5fe90582e1b100"
format_stamp: "Formatted at 2026-08-06 03:51:39 on dist-test-slave-bqcl"
server_key: "51fe63398728c2647b1051e9639ca6b5"
server_key_iv: "029170bc0d628721fdbbf5567bd6df0d"
server_key_version: "encryptionkey@0"
I20260806 03:51:39.248644 13739 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:39.257104 13739 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:39.259125 13739 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:39.259243 13739 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:39.259424 13739 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:39.259678 13739 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:39.259708 13739 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:39.259727 13739 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:39.259744 13739 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:39.265262 13739 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:44173
I20260806 03:51:39.265309 13950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:44173 every 8 connection(s)
I20260806 03:51:39.265683 13739 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988299, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:39.274225 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.266465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:56809 (local address 127.12.15.126:43497)
0806 03:51:39.266523 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.266525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.266718 (+ 193us) server_negotiation.cc:407] Connection header received
0806 03:51:39.267640 (+ 922us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.267641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.267655 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.267719 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:39.268356 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.268861 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.269842 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.270031 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.270086 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.272056 (+ 1970us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:39.272058 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:39.272058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:39.272064 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:39.273144 (+ 1080us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.273479 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.273480 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.273480 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.273527 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.273704 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.273705 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.273705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.273797 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:39.273840 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.274043 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.274106 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 03:51:39.274367 13955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.266432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:43497 (local address 127.12.15.67:56809)
0806 03:51:39.266597 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:39.266634 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:39.267531 (+ 897us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:39.267805 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:39.267809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:39.268059 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:39.268260 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.268265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.268939 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.268940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.269755 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.269758 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.270096 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.270097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.270242 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.270657 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:39.270665 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:39.271965 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:39.273213 (+ 1248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:39.273219 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:39.273221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:39.273424 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:39.273571 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:39.273571 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:39.273571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:39.273659 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:39.273837 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:39.273840 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:39.273946 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 03:51:39.274021 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:51:39.274172 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":77,"threads_started":1}
I20260806 03:51:39.274786 13951 heartbeater.cc:347] Connected to a master server at 127.12.15.126:43497
I20260806 03:51:39.274864 13951 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:39.274924 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13739
I20260806 03:51:39.275025 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:39.275043 13951 heartbeater.cc:511] Master 127.12.15.126:43497 requested a full tablet report, sending...
I20260806 03:51:39.275185 12349 external_mini_cluster.cc:1428] Setting key 7bd44913ad02e84e513a7bc349b68c9f
I20260806 03:51:39.275413 13240 ts_manager.cc:194] Registered new tserver with Master: 944a801e66724c0cab5fe90582e1b100 (127.12.15.67:44173)
I20260806 03:51:39.275802 13240 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:56809
I20260806 03:51:39.276038 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:51:39.283253 13298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.278109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38268 (local address 127.12.15.126:43497)
0806 03:51:39.278208 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.278210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.278280 (+ 70us) server_negotiation.cc:407] Connection header received
0806 03:51:39.278333 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.278334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.278342 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.278392 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:39.278908 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.279392 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.280123 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.280319 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.280376 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.281358 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:39.281359 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:39.281359 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:39.281363 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:39.282374 (+ 1011us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.282642 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.282642 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.282643 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.282689 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.282859 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.282860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.282860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.282983 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:39.283054 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.283124 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.283189 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260806 03:51:39.285583 13240 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:51:39.287392 13240 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:39.294314 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.289786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:39961 (local address 127.0.0.1:34426)
0806 03:51:39.289909 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:39.289948 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:39.290012 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:39.290429 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:39.290433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:39.290438 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:39.290537 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.290539 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.291838 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.291839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.293222 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.293224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.294004 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.294005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.294069 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.294074 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:51:39.294102 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 03:51:39.294199 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":66,"thread_start_us":77,"threads_started":1}
I20260806 03:51:39.294483 13974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.289829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:33607 (local address 127.0.0.1:40794)
0806 03:51:39.290286 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:39.290293 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:39.290327 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:39.290682 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:39.290682 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:39.290684 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:39.290723 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.290723 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.291868 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.291869 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.293199 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.293202 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.293957 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.293958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.294041 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.294055 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:51:39.294195 (+ 140us) client_negotiation.cc:240] Negotiation successful
0806 03:51:39.294241 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"mutex_wait_us":59,"thread_start_us":38,"threads_started":1}
I20260806 03:51:39.294654 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.290032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40794 (local address 127.12.15.66:33607)
0806 03:51:39.290547 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.290550 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.290557 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:51:39.290590 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.290592 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.290606 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.290678 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:39.290756 (+ 78us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.291768 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.293298 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.293887 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.293947 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.294008 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.294346 (+ 338us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.294390 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":115,"threads_started":1}
I20260806 03:51:39.294729 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.290063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34426 (local address 127.12.15.65:39961)
0806 03:51:39.290284 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.290288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.290296 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:39.290331 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.290333 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.290346 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.290414 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:39.290606 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.291743 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.293306 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.293907 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.294518 (+ 611us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.294580 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.294604 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.294644 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":138,"threads_started":1}
I20260806 03:51:39.294723 13973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.289796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:44173 (local address 127.0.0.1:43498)
0806 03:51:39.290106 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:39.290114 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:39.290175 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:39.290429 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:39.290433 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:39.290434 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:39.290537 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.290540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.292598 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.292599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.293515 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.293518 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.294496 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.294497 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.294556 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.294559 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:39.294605 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:51:39.294639 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":22,"thread_start_us":62,"threads_started":1}
I20260806 03:51:39.295006 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.290031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.12.15.67:44173)
0806 03:51:39.290259 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.290264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.290275 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:51:39.290311 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.290313 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.290327 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.290415 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:39.291869 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.292507 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.293599 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.294339 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.294401 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.294461 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.294889 (+ 428us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.294930 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":114,"threads_started":1}
I20260806 03:51:39.295399 13634 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6a7adbe41749668d87f4ca8a13a70d (DEFAULT_TABLE table=test-table [id=954923de610a4e2fb180b7d6c49a0be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:39.295403 13442 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6a7adbe41749668d87f4ca8a13a70d (DEFAULT_TABLE table=test-table [id=954923de610a4e2fb180b7d6c49a0be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:39.295734 13634 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6a7adbe41749668d87f4ca8a13a70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:39.295739 13442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6a7adbe41749668d87f4ca8a13a70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:39.298277 13979 tablet_bootstrap.cc:492] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438: Bootstrap starting.
I20260806 03:51:39.298672 13839 tablet_service.cc:1467] Processing CreateTablet for tablet 7e6a7adbe41749668d87f4ca8a13a70d (DEFAULT_TABLE table=test-table [id=954923de610a4e2fb180b7d6c49a0be1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:39.298795 13979 tablet_bootstrap.cc:654] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:39.298938 13839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e6a7adbe41749668d87f4ca8a13a70d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:39.299376 13979 log.cc:824] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:39.300074 13979 tablet_bootstrap.cc:492] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438: No bootstrap required, opened a new log
I20260806 03:51:39.300132 13979 ts_tablet_manager.cc:1397] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:51:39.300634 13981 tablet_bootstrap.cc:492] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e: Bootstrap starting.
I20260806 03:51:39.301378 13981 tablet_bootstrap.cc:654] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:39.301865 13979 raft_consensus.cc:348] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.302023 13979 raft_consensus.cc:374] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:39.302059 13979 raft_consensus.cc:729] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de422f064a964722b2b79f6eff551438, State: Initialized, Role: FOLLOWER
I20260806 03:51:39.302228 13981 log.cc:824] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:39.302246 13979 consensus_queue.cc:260] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.302510 13979 ts_tablet_manager.cc:1428] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:51:39.302588 13733 heartbeater.cc:503] Master 127.12.15.126:43497 was elected leader, sending a full tablet report...
I20260806 03:51:39.302729 13982 tablet_bootstrap.cc:492] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100: Bootstrap starting.
I20260806 03:51:39.303005 13981 tablet_bootstrap.cc:492] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e: No bootstrap required, opened a new log
I20260806 03:51:39.303076 13981 ts_tablet_manager.cc:1397] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 03:51:39.303385 13982 tablet_bootstrap.cc:654] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:39.303910 13982 log.cc:824] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:39.304644 13982 tablet_bootstrap.cc:492] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100: No bootstrap required, opened a new log
I20260806 03:51:39.304728 13982 ts_tablet_manager.cc:1397] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:39.305152 13981 raft_consensus.cc:348] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.305366 13981 raft_consensus.cc:374] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:39.305413 13981 raft_consensus.cc:729] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aee30ce814a0478b8232f99906c9a84e, State: Initialized, Role: FOLLOWER
I20260806 03:51:39.305529 13981 consensus_queue.cc:260] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.305814 13535 heartbeater.cc:503] Master 127.12.15.126:43497 was elected leader, sending a full tablet report...
I20260806 03:51:39.305824 13981 ts_tablet_manager.cc:1428] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:51:39.306591 13982 raft_consensus.cc:348] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.306749 13982 raft_consensus.cc:374] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:39.306794 13982 raft_consensus.cc:729] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 944a801e66724c0cab5fe90582e1b100, State: Initialized, Role: FOLLOWER
I20260806 03:51:39.306901 13982 consensus_queue.cc:260] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.307135 13982 ts_tablet_manager.cc:1428] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:51:39.307156 13951 heartbeater.cc:503] Master 127.12.15.126:43497 was elected leader, sending a full tablet report...
W20260806 03:51:39.373926 13735 tablet.cc:2369] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:51:39.437007 13991 raft_consensus.cc:482] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:39.437135 13991 raft_consensus.cc:504] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.437726 13991 leader_election.cc:288] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 944a801e66724c0cab5fe90582e1b100 (127.12.15.67:44173), de422f064a964722b2b79f6eff551438 (127.12.15.66:33607)
I20260806 03:51:39.440412 13975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.437958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:49913 (local address 127.12.15.67:44173)
0806 03:51:39.438000 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.438002 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.438035 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:51:39.438109 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.438110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.438142 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.438206 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:39.438344 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.438802 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.439738 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.440100 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.440171 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.440181 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.440308 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.440343 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:39.440412 13537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.437907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:44173 (local address 127.12.15.65:49913)
0806 03:51:39.437985 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:39.437996 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:39.438044 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:39.438216 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:39.438217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:39.438218 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:39.438285 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.438287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.438869 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.438870 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.439639 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.439641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.440170 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.440171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.440248 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.440251 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:39.440295 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:51:39.440338 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":21}
I20260806 03:51:39.440481 13976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.437979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:53989 (local address 127.12.15.66:33607)
0806 03:51:39.438025 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.438026 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.438181 (+ 155us) server_negotiation.cc:407] Connection header received
0806 03:51:39.438240 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.438241 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.438253 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.438300 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:39.438438 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.438918 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.439815 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.440183 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.440246 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.440254 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.440380 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.440415 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:51:39.440527 13994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.437916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:33607 (local address 127.12.15.65:53989)
0806 03:51:39.438093 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:39.438102 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:39.438178 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:39.438307 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:39.438308 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:39.438310 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:39.438383 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.438384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.438990 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.438991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.439734 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:39.439736 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.440258 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.440259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:39.440322 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.440326 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:51:39.440370 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:51:39.440407 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":2,"thread_start_us":119,"threads_started":1}
I20260806 03:51:39.440855 13871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6a7adbe41749668d87f4ca8a13a70d" candidate_uuid: "aee30ce814a0478b8232f99906c9a84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944a801e66724c0cab5fe90582e1b100" is_pre_election: true
I20260806 03:51:39.441042 13871 raft_consensus.cc:2454] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate aee30ce814a0478b8232f99906c9a84e in term 0.
I20260806 03:51:39.441336 13365 leader_election.cc:302] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 944a801e66724c0cab5fe90582e1b100, aee30ce814a0478b8232f99906c9a84e; no voters:
I20260806 03:51:39.441473 13991 raft_consensus.cc:2781] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:39.441530 13991 raft_consensus.cc:482] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:39.441545 13991 raft_consensus.cc:3037] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:39.442267 13991 raft_consensus.cc:504] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.442379 13991 leader_election.cc:288] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [CANDIDATE]: Term 1 election: Requested vote from peers 944a801e66724c0cab5fe90582e1b100 (127.12.15.67:44173), de422f064a964722b2b79f6eff551438 (127.12.15.66:33607)
I20260806 03:51:39.442528 13665 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6a7adbe41749668d87f4ca8a13a70d" candidate_uuid: "aee30ce814a0478b8232f99906c9a84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de422f064a964722b2b79f6eff551438"
I20260806 03:51:39.442538 13871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6a7adbe41749668d87f4ca8a13a70d" candidate_uuid: "aee30ce814a0478b8232f99906c9a84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "944a801e66724c0cab5fe90582e1b100"
I20260806 03:51:39.442628 13665 raft_consensus.cc:3037] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:39.442622 13871 raft_consensus.cc:3037] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:39.440869 13669 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e6a7adbe41749668d87f4ca8a13a70d" candidate_uuid: "aee30ce814a0478b8232f99906c9a84e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de422f064a964722b2b79f6eff551438" is_pre_election: true
I20260806 03:51:39.443351 13665 raft_consensus.cc:2454] T 7e6a7adbe41749668d87f4ca8a13a70d P de422f064a964722b2b79f6eff551438 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aee30ce814a0478b8232f99906c9a84e in term 1.
I20260806 03:51:39.443526 13871 raft_consensus.cc:2454] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate aee30ce814a0478b8232f99906c9a84e in term 1.
I20260806 03:51:39.443567 13368 leader_election.cc:302] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: aee30ce814a0478b8232f99906c9a84e, de422f064a964722b2b79f6eff551438; no voters:
I20260806 03:51:39.443804 13991 raft_consensus.cc:2781] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:39.443907 13991 raft_consensus.cc:686] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [term 1 LEADER]: Becoming Leader. State: Replica: aee30ce814a0478b8232f99906c9a84e, State: Running, Role: LEADER
I20260806 03:51:39.443991 13991 consensus_queue.cc:237] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } }
I20260806 03:51:39.444976 13240 catalog_manager.cc:5495] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e reported cstate change: term changed from 0 to 1, leader changed from <none> to aee30ce814a0478b8232f99906c9a84e (127.12.15.65). New cstate: current_term: 1 leader_uuid: "aee30ce814a0478b8232f99906c9a84e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aee30ce814a0478b8232f99906c9a84e" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39961 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "944a801e66724c0cab5fe90582e1b100" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 44173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de422f064a964722b2b79f6eff551438" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 33607 } health_report { overall_health: UNKNOWN } } }
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988298, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
W20260806 03:51:39.476016 13536 tablet.cc:2369] T 7e6a7adbe41749668d87f4ca8a13a70d P aee30ce814a0478b8232f99906c9a84e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:51:39.477753 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.470174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34438 (local address 127.12.15.65:39961)
0806 03:51:39.470246 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.470248 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.470368 (+ 120us) server_negotiation.cc:407] Connection header received
0806 03:51:39.470430 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.470431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.470445 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.470495 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:39.471065 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.471543 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.472256 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.472445 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.472500 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.474921 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:39.474936 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:39.474937 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:39.474943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:39.476557 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.476971 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.476975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.476976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.477047 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.477206 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.477207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.477207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.477348 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:39.477466 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.477498 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.477620 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988299, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:51:39.500502 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.494420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34452 (local address 127.12.15.65:39961)
0806 03:51:39.494465 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.494467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.494524 (+ 57us) server_negotiation.cc:407] Connection header received
0806 03:51:39.494573 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.494574 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.494583 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.494635 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:39.495147 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.495624 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.496289 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.496489 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.496551 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.498514 (+ 1963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:39.498516 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:39.498516 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:39.498523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:39.499692 (+ 1169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.499968 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.499969 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.499969 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.500014 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.500134 (+ 120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.500134 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.500135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.500214 (+ 79us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:39.500281 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.500382 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.500436 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260806 03:51:39.516249 13954 tablet.cc:2369] T 7e6a7adbe41749668d87f4ca8a13a70d P 944a801e66724c0cab5fe90582e1b100: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[13169](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988299, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:51:39.522581 13977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:39.515953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34458 (local address 127.12.15.65:39961)
0806 03:51:39.516013 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:39.516016 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:39.516152 (+ 136us) server_negotiation.cc:407] Connection header received
0806 03:51:39.516185 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:39.516186 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:39.516198 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:39.516256 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:39.516868 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.517375 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.518231 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:39.518419 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:39.518470 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:39.520600 (+ 2130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:39.520601 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:39.520601 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:39.520608 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:39.521712 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.522000 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.522001 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.522001 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.522048 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:39.522208 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:39.522209 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:39.522209 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:39.522300 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:39.522374 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:39.522454 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 03:51:39.522511 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 03:51:39.523353 13407 tablet_service.cc:2908] Rejecting scan request for tablet 7e6a7adbe41749668d87f4ca8a13a70d: Uninitialized: clean time has not yet been initialized
I20260806 03:51:39.524251 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13337
I20260806 03:51:39.535650 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13540
I20260806 03:51:39.551846 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13739
I20260806 03:51:39.559749 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13190
2026-08-06T03:51:39Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2185 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[14021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[14021](info): set up 2 sockets
Aug 06 03:51:39 dist-test-slave-bqcl krb5kdc[14021](info): commencing operation
krb5kdc: starting...
W20260806 03:51:40.696725 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:40 dist-test-slave-bqcl krb5kdc[14021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:40.814901 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35689
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:35689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:40.893977 14038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:40.894198 14038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:40.894222 14038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:40.895273 14038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:40.895303 14038 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 03:51:40.895314 14038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:40.895326 14038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:40.895337 14038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:40.895347 14038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:40.896688 14038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:35689
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:40.896862 14038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:40.897022 14038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:40.901499 14049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:40.901484 14046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:40.901489 14043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:40.901593 14038 server_base.cc:1034] running on GCE node
I20260806 03:51:40.901902 14038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:40.902173 14038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:40.903309 14038 hybrid_clock.cc:648] HybridClock initialized: now 1785988300903291 us; error 35 us; skew 500 ppm
Aug 06 03:51:40 dist-test-slave-bqcl krb5kdc[14021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988300, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:40.909580 14038 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:40.909956 14038 webserver.cc:460] Webserver started at http://127.12.15.126:33737/ using document root <none> and password file <none>
I20260806 03:51:40.910188 14038 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:40.910236 14038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:40.910331 14038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:40.911276 14038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "cc5257cbd0574721952a9ae069e310b4"
format_stamp: "Formatted at 2026-08-06 03:51:40 on dist-test-slave-bqcl"
server_key: "9f2100da646421b7c6388a591d63a813"
server_key_iv: "475b555da6198613c75af5c17dd27bf6"
server_key_version: "encryptionkey@0"
I20260806 03:51:40.911563 14038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "cc5257cbd0574721952a9ae069e310b4"
format_stamp: "Formatted at 2026-08-06 03:51:40 on dist-test-slave-bqcl"
server_key: "9f2100da646421b7c6388a591d63a813"
server_key_iv: "475b555da6198613c75af5c17dd27bf6"
server_key_version: "encryptionkey@0"
I20260806 03:51:40.912698 14038 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:51:40.913317 14061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:40.913455 14038 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:40.913535 14038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "cc5257cbd0574721952a9ae069e310b4"
format_stamp: "Formatted at 2026-08-06 03:51:40 on dist-test-slave-bqcl"
server_key: "9f2100da646421b7c6388a591d63a813"
server_key_iv: "475b555da6198613c75af5c17dd27bf6"
server_key_version: "encryptionkey@0"
I20260806 03:51:40.913600 14038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:40.920744 14038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:40.922308 14038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:40.922427 14038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:40.926946 14038 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:35689
I20260806 03:51:40.927022 14145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:35689 every 8 connection(s)
I20260806 03:51:40.927284 14038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:40.927866 14146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:40.929246 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14038
I20260806 03:51:40.929345 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:40.929510 12349 external_mini_cluster.cc:1428] Setting key b50b2af04e4e0b9dec12a07337498239
I20260806 03:51:40.930250 14146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4: Bootstrap starting.
I20260806 03:51:40.930924 14146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:40.931252 14146 log.cc:824] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:40.931993 14146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4: No bootstrap required, opened a new log
I20260806 03:51:40.934019 14146 raft_consensus.cc:348] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5257cbd0574721952a9ae069e310b4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35689 } }
I20260806 03:51:40.934240 14146 raft_consensus.cc:374] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:40.934301 14146 raft_consensus.cc:729] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cc5257cbd0574721952a9ae069e310b4, State: Initialized, Role: FOLLOWER
I20260806 03:51:40.934439 14146 consensus_queue.cc:260] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5257cbd0574721952a9ae069e310b4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35689 } }
I20260806 03:51:40.934520 14146 raft_consensus.cc:388] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:40.934548 14146 raft_consensus.cc:482] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:40.934584 14146 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:40.935225 14146 raft_consensus.cc:504] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5257cbd0574721952a9ae069e310b4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35689 } }
I20260806 03:51:40.935336 14146 leader_election.cc:302] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cc5257cbd0574721952a9ae069e310b4; no voters:
I20260806 03:51:40.935500 14146 leader_election.cc:288] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:40.935585 14155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:40.935760 14146 sys_catalog.cc:564] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:40.935791 14155 raft_consensus.cc:686] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [term 1 LEADER]: Becoming Leader. State: Replica: cc5257cbd0574721952a9ae069e310b4, State: Running, Role: LEADER
I20260806 03:51:40.935986 14155 consensus_queue.cc:237] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5257cbd0574721952a9ae069e310b4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35689 } }
Aug 06 03:51:40 dist-test-slave-bqcl krb5kdc[14021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988300, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:40.936372 14155 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cc5257cbd0574721952a9ae069e310b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5257cbd0574721952a9ae069e310b4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35689 } } }
I20260806 03:51:40.936564 14155 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:40.936772 14155 sys_catalog.cc:455] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cc5257cbd0574721952a9ae069e310b4. Latest consensus state: current_term: 1 leader_uuid: "cc5257cbd0574721952a9ae069e310b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cc5257cbd0574721952a9ae069e310b4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35689 } } }
I20260806 03:51:40.936841 14155 sys_catalog.cc:458] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:40.936977 14172 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:40.937465 14172 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:40.940143 14172 catalog_manager.cc:1347] Generated new cluster ID: 904ad895787f4fc5a99e9bf429c299dd
I20260806 03:51:40.940198 14172 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:40.940785 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:40.929997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57930 (local address 127.12.15.126:35689)
0806 03:51:40.930173 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:40.930182 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:40.930207 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:51:40.930628 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:40.930635 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:40.930683 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:40.930794 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:40.931544 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:40.932538 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:40.933591 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:40.933962 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:40.934030 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:40.936926 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:40.936930 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:40.936930 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:40.936943 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:40.939009 (+ 2066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:40.939395 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:40.939398 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:40.939399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:40.939510 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:40.939730 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:40.939731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:40.939731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:40.940028 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:40.940172 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:40.940417 (+ 245us) server_negotiation.cc:299] Negotiation successful
0806 03:51:40.940591 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":60,"threads_started":1}
I20260806 03:51:40.948381 14172 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:40.948871 14172 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:40.952988 14172 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4: Generated new TSK 0
I20260806 03:51:40.953154 14172 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:51:40.983126 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:35689
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:41.086735 14192 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:41.086938 14192 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:41.086961 14192 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:41.088109 14192 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:41.088150 14192 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:41.088176 14192 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:51:41.088192 14192 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:41.089627 14192 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:35689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14192
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:41.089854 14192 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:41.090060 14192 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:41.094910 14202 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:41.094942 14198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:41.094913 14200 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:41.094986 14192 server_base.cc:1034] running on GCE node
I20260806 03:51:41.095248 14192 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:41.095453 14192 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:41.096590 14192 hybrid_clock.cc:648] HybridClock initialized: now 1785988301096573 us; error 27 us; skew 500 ppm
Aug 06 03:51:41 dist-test-slave-bqcl krb5kdc[14021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:41.099002 14192 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:51:41.099439 14192 webserver.cc:460] Webserver started at http://127.12.15.65:39827/ using document root <none> and password file <none>
I20260806 03:51:41.099645 14192 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:41.099696 14192 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:41.099824 14192 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:41.100837 14192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "92cec45e22e34a3eac634d313611b9a7"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "fe3c3b547c64936fbdb782afc9fc1590"
server_key_iv: "09d2a536a895df10f1e978fd5b59a58d"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.101179 14192 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "92cec45e22e34a3eac634d313611b9a7"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "fe3c3b547c64936fbdb782afc9fc1590"
server_key_iv: "09d2a536a895df10f1e978fd5b59a58d"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.102437 14192 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:41.103094 14216 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.103235 14192 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.103320 14192 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "92cec45e22e34a3eac634d313611b9a7"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "fe3c3b547c64936fbdb782afc9fc1590"
server_key_iv: "09d2a536a895df10f1e978fd5b59a58d"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.103380 14192 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:41.114997 14192 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:41.116973 14192 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:41.117117 14192 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:41.117671 14192 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:41.117725 14192 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.117754 14192 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:41.117776 14192 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.124712 14192 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:40061
I20260806 03:51:41.124818 14395 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:40061 every 8 connection(s)
I20260806 03:51:41.125085 14192 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:51:41 dist-test-slave-bqcl krb5kdc[14021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:41.127995 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.118204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:32783 (local address 127.12.15.126:35689)
0806 03:51:41.118270 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.118272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.118730 (+ 458us) server_negotiation.cc:407] Connection header received
0806 03:51:41.119565 (+ 835us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.119567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.119582 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.119640 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:41.120530 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.121302 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.122838 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.123103 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.123156 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.125691 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:41.125693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:41.125693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:41.125699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:41.126981 (+ 1282us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.127306 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.127307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.127307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.127353 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.127520 (+ 167us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.127520 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.127521 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.127615 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:41.127659 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.127854 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.127927 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:51:41.128127 14227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.118333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35689 (local address 127.12.15.65:32783)
0806 03:51:41.118591 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.118631 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.119293 (+ 662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.119763 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.119769 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.120116 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:41.120405 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.120412 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.121399 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.121400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.122733 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.122736 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.123186 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.123187 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.123336 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.123874 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:41.123885 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:41.125608 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:41.127050 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.127054 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.127055 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.127254 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.127395 (+ 141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.127396 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.127396 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.127480 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.127654 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:41.127657 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:41.127768 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.127842 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.127954 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":80,"threads_started":1}
I20260806 03:51:41.128721 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14192
I20260806 03:51:41.128832 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:41.129004 12349 external_mini_cluster.cc:1428] Setting key d416117e564eb945979da885e3d63fba
I20260806 03:51:41.129185 14396 heartbeater.cc:347] Connected to a master server at 127.12.15.126:35689
I20260806 03:51:41.129279 14396 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:41.129463 14396 heartbeater.cc:511] Master 127.12.15.126:35689 requested a full tablet report, sending...
I20260806 03:51:41.129994 14083 ts_manager.cc:194] Registered new tserver with Master: 92cec45e22e34a3eac634d313611b9a7 (127.12.15.65:40061)
I20260806 03:51:41.130625 14083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:32783
I20260806 03:51:41.137488 14405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.129808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35689 (local address 127.12.15.65:51681)
0806 03:51:41.129938 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.129948 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.129993 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.132468 (+ 2475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.132469 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.132624 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:41.132674 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.132675 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.133315 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.133316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.133838 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.133839 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.134250 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.134251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.134320 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.134697 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:41.134700 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:41.134954 (+ 254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:41.136525 (+ 1571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.136526 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.136526 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.136779 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.136970 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.136970 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.136971 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.137022 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.137258 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:41.137258 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:41.137291 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.137335 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.137421 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":78,"threads_started":1}
I20260806 03:51:41.137526 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.130792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:51681 (local address 127.12.15.126:35689)
0806 03:51:41.130835 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.130837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.130845 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:41.130864 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.130865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.130875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.130928 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:41.132760 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.133227 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.133917 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.134183 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.134248 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.135034 (+ 786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:41.135035 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:41.135036 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:41.135043 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:41.136453 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.136843 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.136844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.136845 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.136909 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.137070 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.137070 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.137071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.137198 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:41.137273 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.137362 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.137439 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:51:41.156365 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:35689
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:41.236584 14408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:41.236797 14408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:41.236829 14408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:41.237979 14408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:41.238024 14408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:41.238051 14408 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:51:41.238075 14408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:41.239503 14408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:35689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:41.239748 14408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:41.239959 14408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:41.244748 14415 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:41.244706 14416 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:41.244714 14418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:41.245410 14408 server_base.cc:1034] running on GCE node
I20260806 03:51:41.245615 14408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:41.245837 14408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:41.246973 14408 hybrid_clock.cc:648] HybridClock initialized: now 1785988301246952 us; error 35 us; skew 500 ppm
Aug 06 03:51:41 dist-test-slave-bqcl krb5kdc[14021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:41.249224 14408 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:51:41.249647 14408 webserver.cc:460] Webserver started at http://127.12.15.66:34601/ using document root <none> and password file <none>
I20260806 03:51:41.249852 14408 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:41.249898 14408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:41.250027 14408 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:41.251096 14408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "7f4ef549ec504f62aa55d9c2768dc3a6"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "301cd69fbd4ecdf0c2839211236578c9"
server_key_iv: "3e9aa33135de4b89922455253e57c938"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.251509 14408 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "7f4ef549ec504f62aa55d9c2768dc3a6"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "301cd69fbd4ecdf0c2839211236578c9"
server_key_iv: "3e9aa33135de4b89922455253e57c938"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.252830 14408 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:51:41.253512 14428 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.253686 14408 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:41.253770 14408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "7f4ef549ec504f62aa55d9c2768dc3a6"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "301cd69fbd4ecdf0c2839211236578c9"
server_key_iv: "3e9aa33135de4b89922455253e57c938"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.253829 14408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:41.285833 14408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:41.287668 14408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:41.287809 14408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:41.288326 14408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:41.288367 14408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.288398 14408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:41.288415 14408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:51:41 dist-test-slave-bqcl krb5kdc[14021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:41.295262 14408 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:44897
I20260806 03:51:41.295343 14608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:44897 every 8 connection(s)
I20260806 03:51:41.295678 14408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:51:41.297698 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.288789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:35929 (local address 127.12.15.126:35689)
0806 03:51:41.288843 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.288845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.289049 (+ 204us) server_negotiation.cc:407] Connection header received
0806 03:51:41.289524 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.289525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.289540 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.289614 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:41.290366 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.290903 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.292074 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.292287 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.292343 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.295424 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:41.295426 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:41.295427 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:41.295434 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:41.296695 (+ 1261us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.297029 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.297029 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.297029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.297075 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.297250 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.297250 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.297250 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.297345 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:41.297391 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.297577 (+ 186us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.297630 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:51:41.297868 14439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.288786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35689 (local address 127.12.15.66:35929)
0806 03:51:41.288934 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.288971 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.289426 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.289691 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.289695 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.290006 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:41.290275 (+ 269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.290281 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.290993 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.290993 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.291990 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.291992 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.292357 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.292358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.292484 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.292969 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:41.292981 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:41.295341 (+ 2360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:41.296763 (+ 1422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.296768 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.296769 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.296973 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.297117 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.297118 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.297118 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.297208 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.297386 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:41.297389 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:41.297491 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.297565 (+ 74us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.297674 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":87,"threads_started":1}
I20260806 03:51:41.298333 14610 heartbeater.cc:347] Connected to a master server at 127.12.15.126:35689
I20260806 03:51:41.298424 14610 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:41.298594 14610 heartbeater.cc:511] Master 127.12.15.126:35689 requested a full tablet report, sending...
I20260806 03:51:41.298938 14083 ts_manager.cc:194] Registered new tserver with Master: 7f4ef549ec504f62aa55d9c2768dc3a6 (127.12.15.66:44897)
I20260806 03:51:41.299415 14083 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:35929
I20260806 03:51:41.301229 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14408
I20260806 03:51:41.301321 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:41.301508 12349 external_mini_cluster.cc:1428] Setting key 1a36fcb59764e7dae8a9b83b094f52e3
I20260806 03:51:41.304299 14621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.299237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35689 (local address 127.12.15.66:36517)
0806 03:51:41.299348 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.299357 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.299417 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.299549 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.299550 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.299744 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:41.299806 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.299807 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.300505 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.300506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.301011 (+ 505us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.301012 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.301392 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.301393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.301456 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.301869 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:41.301873 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:41.302205 (+ 332us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:41.303385 (+ 1180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.303386 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.303387 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.303539 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.303705 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.303706 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.303706 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.303761 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.304006 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:41.304007 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:41.304050 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.304110 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.304206 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":1,"thread_start_us":62,"threads_started":1}
I20260806 03:51:41.304296 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.299316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:36517 (local address 127.12.15.126:35689)
0806 03:51:41.299353 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.299354 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.299403 (+ 49us) server_negotiation.cc:407] Connection header received
0806 03:51:41.299467 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.299468 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.299476 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.299530 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:41.299871 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.300428 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.301067 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.301316 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.301379 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.302269 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:41.302270 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:41.302271 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:41.302277 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:41.303317 (+ 1040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.303596 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.303597 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.303598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.303650 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.303810 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.303811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.303811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.303948 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:41.304005 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.304138 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.304213 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:51:41.324735 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:35689
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38897
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:41.404403 14624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:41.404604 14624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:41.404626 14624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:41.405732 14624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:41.405766 14624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:41.405786 14624 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:51:41.405805 14624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:41.407244 14624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38897
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:35689
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:41.407462 14624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:41.407675 14624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:41.412427 14638 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:41.412436 14634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:41.412495 14633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:41.412652 14624 server_base.cc:1034] running on GCE node
I20260806 03:51:41.412849 14624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:41.413066 14624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:41.414165 14624 hybrid_clock.cc:648] HybridClock initialized: now 1785988301414142 us; error 47 us; skew 500 ppm
Aug 06 03:51:41 dist-test-slave-bqcl krb5kdc[14021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:41.416496 14624 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:51:41.416966 14624 webserver.cc:460] Webserver started at http://127.12.15.67:33159/ using document root <none> and password file <none>
I20260806 03:51:41.417198 14624 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:41.417248 14624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:41.417377 14624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:41.418407 14624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "af649f700e8648caa447c0cdd08ab01e"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "ecc5a09ca82a0ebd34fb4bc2bdaed3a7"
server_key_iv: "280e58a88725f1ab1f6a1d6536614342"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.418730 14624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "af649f700e8648caa447c0cdd08ab01e"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "ecc5a09ca82a0ebd34fb4bc2bdaed3a7"
server_key_iv: "280e58a88725f1ab1f6a1d6536614342"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.419956 14624 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:41.420579 14647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.420711 14624 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:41.420794 14624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "af649f700e8648caa447c0cdd08ab01e"
format_stamp: "Formatted at 2026-08-06 03:51:41 on dist-test-slave-bqcl"
server_key: "ecc5a09ca82a0ebd34fb4bc2bdaed3a7"
server_key_iv: "280e58a88725f1ab1f6a1d6536614342"
server_key_version: "encryptionkey@0"
I20260806 03:51:41.420853 14624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:41.434778 14624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:41.436573 14624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:41.436702 14624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:41.437232 14624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:41.437282 14624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.437311 14624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:41.437326 14624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.443609 14624 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:39085
Aug 06 03:51:41 dist-test-slave-bqcl krb5kdc[14021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:41.443666 14822 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:39085 every 8 connection(s)
I20260806 03:51:41.444007 14624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:51:41.446403 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.437594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:59665 (local address 127.12.15.126:35689)
0806 03:51:41.437645 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.437647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.437848 (+ 201us) server_negotiation.cc:407] Connection header received
0806 03:51:41.438341 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.438342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.438356 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.438433 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:41.439154 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.439903 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.441347 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.441637 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.441715 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.444209 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:41.444211 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:41.444211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:41.444220 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:41.445329 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.445659 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.445660 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.445660 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.445715 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.445892 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.445893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.445893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.445987 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:41.446031 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.446275 (+ 244us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.446339 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:51:41.446563 14661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.437578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35689 (local address 127.12.15.67:59665)
0806 03:51:41.437724 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.437763 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.438251 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.438485 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.438489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.438798 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:41.439065 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.439071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.440002 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.440003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.441254 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.441257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.441755 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.441756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.441898 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.442388 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:41.442396 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:41.444114 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:41.445395 (+ 1281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.445400 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.445402 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.445604 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.445759 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.445760 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.445760 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.445850 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.446028 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:41.446030 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:41.446158 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.446255 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.446369 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":74,"threads_started":1}
I20260806 03:51:41.447024 14824 heartbeater.cc:347] Connected to a master server at 127.12.15.126:35689
I20260806 03:51:41.447106 14824 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:41.447283 14824 heartbeater.cc:511] Master 127.12.15.126:35689 requested a full tablet report, sending...
I20260806 03:51:41.447645 14082 ts_manager.cc:194] Registered new tserver with Master: af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085)
I20260806 03:51:41.448132 14082 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:59665
I20260806 03:51:41.449275 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14624
I20260806 03:51:41.449350 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:41.449502 12349 external_mini_cluster.cc:1428] Setting key c6ef8ab6820024971ed161e89784f98d
I20260806 03:51:41.450424 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:51:41.453229 14835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.447942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35689 (local address 127.12.15.67:57229)
0806 03:51:41.448051 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.448061 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.448119 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.448308 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.448309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.448483 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:41.448540 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.448542 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.449142 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.449143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.449736 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.449738 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.450104 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.450105 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.450204 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.450548 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:41.450552 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:41.450883 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:41.452210 (+ 1327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.452211 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.452211 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.452418 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.452628 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:41.452628 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:41.452628 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:41.452680 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:41.452911 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:41.452912 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:41.452977 (+ 65us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.453052 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.453150 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":59,"threads_started":1}
I20260806 03:51:41.453253 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.448018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:57229 (local address 127.12.15.126:35689)
0806 03:51:41.448057 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.448058 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.448106 (+ 48us) server_negotiation.cc:407] Connection header received
0806 03:51:41.448170 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.448170 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.448178 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.448225 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:41.448626 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.449067 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.449804 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.450039 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.450091 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.450959 (+ 868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:41.450960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:41.450960 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:41.450966 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:41.452131 (+ 1165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.452503 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.452504 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.452505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.452569 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.452729 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.452730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.452730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.452848 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:41.452920 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.453093 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.453164 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 06 03:51:41 dist-test-slave-bqcl krb5kdc[14021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:51:41 dist-test-slave-bqcl krb5kdc[14021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:41.472826 14151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.467019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57942 (local address 127.12.15.126:35689)
0806 03:51:41.467071 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.467072 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.467082 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:51:41.467104 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.467105 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.467113 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.467160 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:41.467660 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.468136 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.468792 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.468960 (+ 168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.469002 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.470968 (+ 1966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:41.470969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:41.470970 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:41.470976 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:41.471998 (+ 1022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.472306 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.472307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.472307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.472350 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:41.472475 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:41.472475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:41.472476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:41.472567 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:41.472634 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.472704 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.472759 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:51:41.474305 14082 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:51:41.475858 14082 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:41.482231 14849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.478081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:39085 (local address 127.0.0.1:46276)
0806 03:51:41.478363 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.478375 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.478426 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.478772 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.478775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.478777 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:41.478847 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.478849 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.480209 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.480210 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.481264 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.481266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.481913 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.481914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.481991 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.482005 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.482070 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.482154 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":124,"threads_started":1}
I20260806 03:51:41.482309 14850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.478028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40061 (local address 127.0.0.1:39382)
0806 03:51:41.478256 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.478299 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.478364 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.478977 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.478978 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.478980 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:41.479032 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.479033 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.480286 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.480287 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.481137 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.481139 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.482035 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.482036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.482092 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.482095 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.482175 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.482218 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":63,"threads_started":1}
I20260806 03:51:41.482414 14852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.478266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:44897 (local address 127.0.0.1:55660)
0806 03:51:41.478585 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.478594 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.478673 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.479023 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.479024 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.479025 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:41.479059 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.479060 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.479915 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.479916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.481195 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.481198 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.482159 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.482160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.482229 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.482234 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.482291 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.482336 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"mutex_wait_us":29,"thread_start_us":63,"threads_started":1}
I20260806 03:51:41.482544 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.478387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55660 (local address 127.12.15.66:44897)
0806 03:51:41.478534 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.478539 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.478657 (+ 118us) server_negotiation.cc:407] Connection header received
0806 03:51:41.478854 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.478858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.478876 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.478952 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:41.479131 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.479837 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.481271 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.481990 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.482047 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.482102 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.482405 (+ 303us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.482450 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260806 03:51:41.482568 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.478405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46276 (local address 127.12.15.67:39085)
0806 03:51:41.478579 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.478584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.478595 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:51:41.478636 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.478639 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.478655 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.478926 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:41.478988 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.480116 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.481323 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.481837 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.481902 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.481964 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.482437 (+ 473us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.482481 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260806 03:51:41.483011 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.478185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39382 (local address 127.12.15.65:40061)
0806 03:51:41.478806 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.478810 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.478819 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:41.478856 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.478858 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.478873 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.478976 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:41.479099 (+ 123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.480179 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.481223 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.481960 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.482039 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.482101 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.482192 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.482234 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":80,"threads_started":1}
I20260806 03:51:41.483215 14729 tablet_service.cc:1467] Processing CreateTablet for tablet b35050db33da4ac7a3802ed4ded7491b (DEFAULT_TABLE table=test-table [id=1d7e7fffebc442458a9040cad1a63f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:41.483242 14509 tablet_service.cc:1467] Processing CreateTablet for tablet b35050db33da4ac7a3802ed4ded7491b (DEFAULT_TABLE table=test-table [id=1d7e7fffebc442458a9040cad1a63f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:41.483584 14729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b35050db33da4ac7a3802ed4ded7491b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:41.483604 14509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b35050db33da4ac7a3802ed4ded7491b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:41.485958 14857 tablet_bootstrap.cc:492] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Bootstrap starting.
I20260806 03:51:41.486411 14859 tablet_bootstrap.cc:492] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Bootstrap starting.
I20260806 03:51:41.486572 14282 tablet_service.cc:1467] Processing CreateTablet for tablet b35050db33da4ac7a3802ed4ded7491b (DEFAULT_TABLE table=test-table [id=1d7e7fffebc442458a9040cad1a63f53]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:41.486905 14282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b35050db33da4ac7a3802ed4ded7491b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:41.487056 14859 tablet_bootstrap.cc:654] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:41.487056 14857 tablet_bootstrap.cc:654] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:41.487337 14857 log.cc:824] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:41.487378 14859 log.cc:824] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:41.488025 14859 tablet_bootstrap.cc:492] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: No bootstrap required, opened a new log
I20260806 03:51:41.488067 14857 tablet_bootstrap.cc:492] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: No bootstrap required, opened a new log
I20260806 03:51:41.488088 14859 ts_tablet_manager.cc:1397] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:51:41.488127 14857 ts_tablet_manager.cc:1397] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:41.489527 14859 raft_consensus.cc:348] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.489660 14859 raft_consensus.cc:374] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:41.489691 14859 raft_consensus.cc:729] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f4ef549ec504f62aa55d9c2768dc3a6, State: Initialized, Role: FOLLOWER
I20260806 03:51:41.489821 14859 consensus_queue.cc:260] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.490072 14859 ts_tablet_manager.cc:1428] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:41.490011 14857 raft_consensus.cc:348] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.490195 14857 raft_consensus.cc:374] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:41.490221 14610 heartbeater.cc:503] Master 127.12.15.126:35689 was elected leader, sending a full tablet report...
I20260806 03:51:41.490234 14857 raft_consensus.cc:729] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af649f700e8648caa447c0cdd08ab01e, State: Initialized, Role: FOLLOWER
I20260806 03:51:41.490335 14857 consensus_queue.cc:260] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.490468 14864 tablet_bootstrap.cc:492] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Bootstrap starting.
I20260806 03:51:41.490577 14857 ts_tablet_manager.cc:1428] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:51:41.490737 14824 heartbeater.cc:503] Master 127.12.15.126:35689 was elected leader, sending a full tablet report...
I20260806 03:51:41.491050 14864 tablet_bootstrap.cc:654] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:41.491376 14864 log.cc:824] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:41.491956 14864 tablet_bootstrap.cc:492] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: No bootstrap required, opened a new log
I20260806 03:51:41.492012 14864 ts_tablet_manager.cc:1397] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:51:41.493314 14864 raft_consensus.cc:348] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.493428 14864 raft_consensus.cc:374] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:41.493458 14864 raft_consensus.cc:729] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92cec45e22e34a3eac634d313611b9a7, State: Initialized, Role: FOLLOWER
I20260806 03:51:41.493557 14864 consensus_queue.cc:260] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.493781 14864 ts_tablet_manager.cc:1428] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:41.493848 14396 heartbeater.cc:503] Master 127.12.15.126:35689 was elected leader, sending a full tablet report...
W20260806 03:51:41.546311 14612 tablet.cc:2369] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:51:41.625851 14397 tablet.cc:2369] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:51:41.694641 14826 tablet.cc:2369] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:51:41.719475 14867 raft_consensus.cc:482] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:41.719610 14867 raft_consensus.cc:504] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.719985 14867 leader_election.cc:288] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085), 7f4ef549ec504f62aa55d9c2768dc3a6 (127.12.15.66:44897)
I20260806 03:51:41.722782 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.720251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:48065 (local address 127.12.15.67:39085)
0806 03:51:41.720312 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.720314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.720373 (+ 59us) server_negotiation.cc:407] Connection header received
0806 03:51:41.720435 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.720436 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.720448 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.720489 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:41.720638 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.721125 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.722079 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.722473 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.722537 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.722546 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.722671 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.722718 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:51:41.722810 14876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.720160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:39085 (local address 127.12.15.65:48065)
0806 03:51:41.720293 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.720304 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.720370 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.720502 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.720503 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.720504 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:41.720577 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.720579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.721204 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.721205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.721994 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.721996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.722546 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.722547 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.722609 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.722612 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.722657 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.722696 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":1,"thread_start_us":69,"threads_started":1}
I20260806 03:51:41.722852 14855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.720254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:47417 (local address 127.12.15.66:44897)
0806 03:51:41.720299 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.720301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.720413 (+ 112us) server_negotiation.cc:407] Connection header received
0806 03:51:41.720470 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.720471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.720480 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.720528 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:41.720666 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.721123 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.722077 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.722502 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.722567 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.722575 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.722749 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.722792 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:41.723008 14870 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.720160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:44897 (local address 127.12.15.65:47417)
0806 03:51:41.720334 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:41.720344 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:41.720414 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:41.720535 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:41.720535 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:41.720538 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:41.720606 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.720608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.721194 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.721194 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.721990 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:41.721993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.722574 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.722574 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:41.722646 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.722649 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:41.722702 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:51:41.722743 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"mutex_wait_us":59,"thread_start_us":67,"threads_started":1}
I20260806 03:51:41.723156 14758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b35050db33da4ac7a3802ed4ded7491b" candidate_uuid: "92cec45e22e34a3eac634d313611b9a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af649f700e8648caa447c0cdd08ab01e" is_pre_election: true
I20260806 03:51:41.723325 14758 raft_consensus.cc:2454] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92cec45e22e34a3eac634d313611b9a7 in term 0.
I20260806 03:51:41.723389 14546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b35050db33da4ac7a3802ed4ded7491b" candidate_uuid: "92cec45e22e34a3eac634d313611b9a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" is_pre_election: true
I20260806 03:51:41.723558 14546 raft_consensus.cc:2454] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92cec45e22e34a3eac634d313611b9a7 in term 0.
I20260806 03:51:41.723559 14218 leader_election.cc:302] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92cec45e22e34a3eac634d313611b9a7, af649f700e8648caa447c0cdd08ab01e; no voters:
I20260806 03:51:41.723748 14867 raft_consensus.cc:2781] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:41.723834 14867 raft_consensus.cc:482] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:41.723870 14867 raft_consensus.cc:3037] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:41.724581 14867 raft_consensus.cc:504] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.724697 14867 leader_election.cc:288] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [CANDIDATE]: Term 1 election: Requested vote from peers af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085), 7f4ef549ec504f62aa55d9c2768dc3a6 (127.12.15.66:44897)
I20260806 03:51:41.724845 14546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b35050db33da4ac7a3802ed4ded7491b" candidate_uuid: "92cec45e22e34a3eac634d313611b9a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6"
I20260806 03:51:41.724845 14758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b35050db33da4ac7a3802ed4ded7491b" candidate_uuid: "92cec45e22e34a3eac634d313611b9a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af649f700e8648caa447c0cdd08ab01e"
I20260806 03:51:41.724929 14546 raft_consensus.cc:3037] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:41.724929 14758 raft_consensus.cc:3037] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:41.725818 14758 raft_consensus.cc:2454] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92cec45e22e34a3eac634d313611b9a7 in term 1.
I20260806 03:51:41.725818 14546 raft_consensus.cc:2454] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92cec45e22e34a3eac634d313611b9a7 in term 1.
I20260806 03:51:41.725991 14217 leader_election.cc:302] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f4ef549ec504f62aa55d9c2768dc3a6, 92cec45e22e34a3eac634d313611b9a7; no voters:
I20260806 03:51:41.726094 14867 raft_consensus.cc:2781] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:41.726224 14867 raft_consensus.cc:686] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 1 LEADER]: Becoming Leader. State: Replica: 92cec45e22e34a3eac634d313611b9a7, State: Running, Role: LEADER
I20260806 03:51:41.726341 14867 consensus_queue.cc:237] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } }
I20260806 03:51:41.727063 14082 catalog_manager.cc:5495] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92cec45e22e34a3eac634d313611b9a7 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } health_report { overall_health: UNKNOWN } } }
I20260806 03:51:41.762816 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:41.760621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.12.15.65:40061)
0806 03:51:41.760670 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:41.760672 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:41.760682 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:51:41.760720 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:41.760721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:41.760740 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:41.760801 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:41.760908 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.761383 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.762118 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:41.762349 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:41.762403 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:41.762549 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:41.762651 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:41.762704 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:41.762731 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 03:51:41.762752 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:51:41.764670 14758 raft_consensus.cc:1261] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Refusing update from remote peer 92cec45e22e34a3eac634d313611b9a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:41.764668 14546 raft_consensus.cc:1261] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Refusing update from remote peer 92cec45e22e34a3eac634d313611b9a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:41.764997 14877 consensus_queue.cc:1035] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:41.765121 14867 consensus_queue.cc:1035] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:41.771992 14083 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:41.772113 14083 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:41.772931 14083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4: Sending DeleteTablet for 3 replicas of tablet b35050db33da4ac7a3802ed4ded7491b
I20260806 03:51:41.773389 14729 tablet_service.cc:1514] Processing DeleteTablet for tablet b35050db33da4ac7a3802ed4ded7491b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:41 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:46276
I20260806 03:51:41.773396 14282 tablet_service.cc:1514] Processing DeleteTablet for tablet b35050db33da4ac7a3802ed4ded7491b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:41 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:39382
I20260806 03:51:41.773399 14509 tablet_service.cc:1514] Processing DeleteTablet for tablet b35050db33da4ac7a3802ed4ded7491b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:41 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:55660
I20260806 03:51:41.773784 14902 tablet_replica.cc:330] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: stopping tablet replica
I20260806 03:51:41.773891 14902 raft_consensus.cc:2229] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:41.773886 14083 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57942:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 03:51:41.773994 14902 raft_consensus.cc:2258] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260806 03:51:41.774029 14083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:41.774183 14901 tablet_replica.cc:330] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: stopping tablet replica
I20260806 03:51:41.774281 14901 raft_consensus.cc:2229] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:41.774634 14901 raft_consensus.cc:2258] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:41.774916 14902 ts_tablet_manager.cc:1905] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:41.775457 14509 tablet_service.cc:1467] Processing CreateTablet for tablet 7684dbacbe674348943080dcee42992d (DEFAULT_TABLE table=test-table [id=5ffe7c36f9dd4a83bafb15c52bf0f257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:41.775585 14509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7684dbacbe674348943080dcee42992d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:41.775709 14729 tablet_service.cc:1467] Processing CreateTablet for tablet 7684dbacbe674348943080dcee42992d (DEFAULT_TABLE table=test-table [id=5ffe7c36f9dd4a83bafb15c52bf0f257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:41.775813 14729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7684dbacbe674348943080dcee42992d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:41.776916 14859 tablet_bootstrap.cc:492] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: Bootstrap starting.
I20260806 03:51:41.777061 14857 tablet_bootstrap.cc:492] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: Bootstrap starting.
I20260806 03:51:41.777228 14282 tablet_service.cc:1467] Processing CreateTablet for tablet 7684dbacbe674348943080dcee42992d (DEFAULT_TABLE table=test-table [id=5ffe7c36f9dd4a83bafb15c52bf0f257]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:41.777383 14282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7684dbacbe674348943080dcee42992d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:41.777478 14857 tablet_bootstrap.cc:654] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:41.777652 14859 tablet_bootstrap.cc:654] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:41.778378 14859 tablet_bootstrap.cc:492] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: No bootstrap required, opened a new log
I20260806 03:51:41.778431 14859 ts_tablet_manager.cc:1397] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:51:41.778391 14901 ts_tablet_manager.cc:1905] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:41.778568 14859 raft_consensus.cc:348] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:41.778651 14859 raft_consensus.cc:374] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:41.778677 14859 raft_consensus.cc:729] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f4ef549ec504f62aa55d9c2768dc3a6, State: Initialized, Role: FOLLOWER
I20260806 03:51:41.778716 14859 consensus_queue.cc:260] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:41.778797 14859 ts_tablet_manager.cc:1428] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:41.778859 14864 tablet_bootstrap.cc:492] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: Bootstrap starting.
I20260806 03:51:41.779269 14864 tablet_bootstrap.cc:654] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:41.779964 14864 tablet_bootstrap.cc:492] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: No bootstrap required, opened a new log
I20260806 03:51:41.780018 14864 ts_tablet_manager.cc:1397] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:41.780161 14864 raft_consensus.cc:348] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:41.780243 14864 raft_consensus.cc:374] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:41.780269 14864 raft_consensus.cc:729] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92cec45e22e34a3eac634d313611b9a7, State: Initialized, Role: FOLLOWER
I20260806 03:51:41.780308 14864 consensus_queue.cc:260] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:41.780397 14864 ts_tablet_manager.cc:1428] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.780575 14901 ts_tablet_manager.cc:1918] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:51:41.780684 14901 log.cc:1196] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/b35050db33da4ac7a3802ed4ded7491b
I20260806 03:51:41.780985 14901 ts_tablet_manager.cc:1939] T b35050db33da4ac7a3802ed4ded7491b P af649f700e8648caa447c0cdd08ab01e: Deleting consensus metadata
I20260806 03:51:41.781003 14902 ts_tablet_manager.cc:1918] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:51:41.781061 14902 log.cc:1196] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/b35050db33da4ac7a3802ed4ded7491b
I20260806 03:51:41.781438 14905 tablet_replica.cc:330] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: stopping tablet replica
I20260806 03:51:41.781391 14064 catalog_manager.cc:4841] TS af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085): tablet b35050db33da4ac7a3802ed4ded7491b (table test-table [id=1d7e7fffebc442458a9040cad1a63f53]) successfully deleted
I20260806 03:51:41.781558 14905 raft_consensus.cc:2229] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:51:41.781615 14902 ts_tablet_manager.cc:1939] T b35050db33da4ac7a3802ed4ded7491b P 7f4ef549ec504f62aa55d9c2768dc3a6: Deleting consensus metadata
I20260806 03:51:41.781665 14905 raft_consensus.cc:2258] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:41.782096 14857 tablet_bootstrap.cc:492] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: No bootstrap required, opened a new log
I20260806 03:51:41.782157 14857 ts_tablet_manager.cc:1397] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 03:51:41.782148 14905 ts_tablet_manager.cc:1905] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:41.782307 14857 raft_consensus.cc:348] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:41.782361 14857 raft_consensus.cc:374] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:41.782378 14857 raft_consensus.cc:729] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af649f700e8648caa447c0cdd08ab01e, State: Initialized, Role: FOLLOWER
I20260806 03:51:41.782418 14857 consensus_queue.cc:260] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:41.782500 14857 ts_tablet_manager.cc:1428] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:41.783557 14905 ts_tablet_manager.cc:1918] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:51:41.783645 14905 log.cc:1196] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/b35050db33da4ac7a3802ed4ded7491b
I20260806 03:51:41.783859 14905 ts_tablet_manager.cc:1939] T b35050db33da4ac7a3802ed4ded7491b P 92cec45e22e34a3eac634d313611b9a7: Deleting consensus metadata
I20260806 03:51:41.784205 14062 catalog_manager.cc:4841] TS 92cec45e22e34a3eac634d313611b9a7 (127.12.15.65:40061): tablet b35050db33da4ac7a3802ed4ded7491b (table test-table [id=1d7e7fffebc442458a9040cad1a63f53]) successfully deleted
I20260806 03:51:41.784377 14062 catalog_manager.cc:4841] TS 7f4ef549ec504f62aa55d9c2768dc3a6 (127.12.15.66:44897): tablet b35050db33da4ac7a3802ed4ded7491b (table test-table [id=1d7e7fffebc442458a9040cad1a63f53]) successfully deleted
I20260806 03:51:42.178382 14863 raft_consensus.cc:482] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:42.178483 14863 raft_consensus.cc:504] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.178791 14863 leader_election.cc:288] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92cec45e22e34a3eac634d313611b9a7 (127.12.15.65:40061), af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085)
I20260806 03:51:42.181584 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.179039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:59903 (local address 127.12.15.65:40061)
0806 03:51:42.179127 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.179129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.179245 (+ 116us) server_negotiation.cc:407] Connection header received
0806 03:51:42.179296 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.179297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.179307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.179360 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:42.179505 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.179951 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.180899 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.181259 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.181319 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.181329 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.181473 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.181512 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260806 03:51:42.181584 14856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.179038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:42081 (local address 127.12.15.67:39085)
0806 03:51:42.179104 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.179106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.179220 (+ 114us) server_negotiation.cc:407] Connection header received
0806 03:51:42.179275 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.179276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.179285 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.179333 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:42.179472 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.179950 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.180911 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.181267 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.181329 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.181337 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.181475 (+ 138us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.181512 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:51:42.181617 14912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.178977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:39085 (local address 127.12.15.66:42081)
0806 03:51:42.179141 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:42.179152 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:42.179214 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:42.179347 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:42.179348 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:42.179351 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:42.179413 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.179415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.180023 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.180024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.180822 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.180825 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.181342 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.181343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.181408 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.181411 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:42.181459 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 03:51:42.181503 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":35,"thread_start_us":82,"threads_started":1}
I20260806 03:51:42.181622 14913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.178977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40061 (local address 127.12.15.66:59903)
0806 03:51:42.179166 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:42.179175 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:42.179243 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:42.179369 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:42.179370 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:42.179372 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:42.179442 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.179444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.180020 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.180022 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.180811 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.180813 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.181330 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.181330 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.181403 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.181408 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:51:42.181455 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:51:42.181499 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"mutex_wait_us":3,"thread_start_us":77,"threads_started":1}
I20260806 03:51:42.181867 14758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7684dbacbe674348943080dcee42992d" candidate_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af649f700e8648caa447c0cdd08ab01e" is_pre_election: true
I20260806 03:51:42.181968 14758 raft_consensus.cc:2454] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f4ef549ec504f62aa55d9c2768dc3a6 in term 0.
I20260806 03:51:42.182075 14317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7684dbacbe674348943080dcee42992d" candidate_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92cec45e22e34a3eac634d313611b9a7" is_pre_election: true
I20260806 03:51:42.182193 14431 leader_election.cc:302] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f4ef549ec504f62aa55d9c2768dc3a6, af649f700e8648caa447c0cdd08ab01e; no voters:
I20260806 03:51:42.182248 14317 raft_consensus.cc:2454] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f4ef549ec504f62aa55d9c2768dc3a6 in term 0.
I20260806 03:51:42.182312 14863 raft_consensus.cc:2781] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:42.182348 14863 raft_consensus.cc:482] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:42.182367 14863 raft_consensus.cc:3037] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:42.183028 14863 raft_consensus.cc:504] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.183138 14863 leader_election.cc:288] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [CANDIDATE]: Term 1 election: Requested vote from peers 92cec45e22e34a3eac634d313611b9a7 (127.12.15.65:40061), af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085)
I20260806 03:51:42.183305 14317 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7684dbacbe674348943080dcee42992d" candidate_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92cec45e22e34a3eac634d313611b9a7"
I20260806 03:51:42.183348 14758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7684dbacbe674348943080dcee42992d" candidate_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af649f700e8648caa447c0cdd08ab01e"
I20260806 03:51:42.183389 14317 raft_consensus.cc:3037] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:42.183410 14758 raft_consensus.cc:3037] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:42.183993 14758 raft_consensus.cc:2454] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f4ef549ec504f62aa55d9c2768dc3a6 in term 1.
I20260806 03:51:42.184039 14317 raft_consensus.cc:2454] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f4ef549ec504f62aa55d9c2768dc3a6 in term 1.
I20260806 03:51:42.184190 14431 leader_election.cc:302] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f4ef549ec504f62aa55d9c2768dc3a6, af649f700e8648caa447c0cdd08ab01e; no voters:
I20260806 03:51:42.184273 14863 raft_consensus.cc:2781] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:42.184365 14863 raft_consensus.cc:686] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 LEADER]: Becoming Leader. State: Replica: 7f4ef549ec504f62aa55d9c2768dc3a6, State: Running, Role: LEADER
I20260806 03:51:42.184422 14863 consensus_queue.cc:237] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.185068 14083 catalog_manager.cc:5495] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f4ef549ec504f62aa55d9c2768dc3a6 (127.12.15.66). New cstate: current_term: 1 leader_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } health_report { overall_health: UNKNOWN } } }
Aug 06 03:51:42 dist-test-slave-bqcl krb5kdc[14021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988300, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:42.241122 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.234751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35689 (local address 127.0.0.1:57958)
0806 03:51:42.234911 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:42.234920 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:42.234974 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:42.235097 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:42.235098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:42.235394 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:42.235442 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.235444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.236102 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.236103 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.236655 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.236657 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.237008 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.237010 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.237081 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.237592 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:42.237596 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:42.238930 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:42.240250 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:42.240256 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:42.240258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:42.240452 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:42.240623 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:42.240623 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:42.240623 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:42.240676 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:42.240852 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:42.240857 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:42.240897 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 03:51:42.240942 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:51:42.241027 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":67,"threads_started":1}
I20260806 03:51:42.241175 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.234793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57958 (local address 127.12.15.126:35689)
0806 03:51:42.234914 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.234915 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.234969 (+ 54us) server_negotiation.cc:407] Connection header received
0806 03:51:42.235013 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.235013 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.235023 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.235088 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:42.235496 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.236048 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.236716 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.236901 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.236945 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.239017 (+ 2072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:42.239019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:42.239019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:42.239025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:42.240187 (+ 1162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:42.240532 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:42.240532 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:42.240533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:42.240580 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:42.240715 (+ 135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:42.240715 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:42.240715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:42.240811 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:42.240856 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.240964 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.241073 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260806 03:51:42.243185 14083 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:57958:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 03:51:42.243432 14083 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:42.244830 14282 tablet_service.cc:1467] Processing CreateTablet for tablet c11cf60e3cdb4e8ca5c8de1eea9736f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1d1b71bc87345809c34e49ad2482456]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:51:42.244863 14729 tablet_service.cc:1467] Processing CreateTablet for tablet c11cf60e3cdb4e8ca5c8de1eea9736f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1d1b71bc87345809c34e49ad2482456]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:51:42.244930 14509 tablet_service.cc:1467] Processing CreateTablet for tablet c11cf60e3cdb4e8ca5c8de1eea9736f2 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=c1d1b71bc87345809c34e49ad2482456]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:51:42.245062 14282 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c11cf60e3cdb4e8ca5c8de1eea9736f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:42.245060 14729 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c11cf60e3cdb4e8ca5c8de1eea9736f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:42.245070 14509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c11cf60e3cdb4e8ca5c8de1eea9736f2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:42.246631 14864 tablet_bootstrap.cc:492] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: Bootstrap starting.
I20260806 03:51:42.246649 14857 tablet_bootstrap.cc:492] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: Bootstrap starting.
I20260806 03:51:42.246649 14859 tablet_bootstrap.cc:492] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: Bootstrap starting.
I20260806 03:51:42.246943 14864 tablet_bootstrap.cc:654] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:42.247083 14857 tablet_bootstrap.cc:654] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:42.247093 14859 tablet_bootstrap.cc:654] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:42.247784 14859 tablet_bootstrap.cc:492] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: No bootstrap required, opened a new log
I20260806 03:51:42.247835 14859 ts_tablet_manager.cc:1397] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:42.247876 14864 tablet_bootstrap.cc:492] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: No bootstrap required, opened a new log
I20260806 03:51:42.247912 14864 ts_tablet_manager.cc:1397] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:42.247977 14857 tablet_bootstrap.cc:492] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: No bootstrap required, opened a new log
I20260806 03:51:42.248019 14857 ts_tablet_manager.cc:1397] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:42.247983 14859 raft_consensus.cc:348] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.247993 14864 raft_consensus.cc:348] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.248047 14859 raft_consensus.cc:374] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:42.248064 14864 raft_consensus.cc:374] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:42.248066 14859 raft_consensus.cc:729] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f4ef549ec504f62aa55d9c2768dc3a6, State: Initialized, Role: FOLLOWER
I20260806 03:51:42.248087 14864 raft_consensus.cc:729] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92cec45e22e34a3eac634d313611b9a7, State: Initialized, Role: FOLLOWER
I20260806 03:51:42.248117 14859 consensus_queue.cc:260] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.248126 14864 consensus_queue.cc:260] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.248152 14857 raft_consensus.cc:348] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.248210 14857 raft_consensus.cc:374] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:42.248214 14859 ts_tablet_manager.cc:1428] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:42.248227 14857 raft_consensus.cc:729] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: af649f700e8648caa447c0cdd08ab01e, State: Initialized, Role: FOLLOWER
I20260806 03:51:42.248236 14864 ts_tablet_manager.cc:1428] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:42.248266 14857 consensus_queue.cc:260] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.248258 14863 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.248345 14857 ts_tablet_manager.cc:1428] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:42.248342 14863 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:42.248323 14895 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.248399 14895 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:42.248482 14865 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.248549 14865 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:42.332758 14895 raft_consensus.cc:482] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:42.332866 14895 raft_consensus.cc:504] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.333039 14895 leader_election.cc:288] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f4ef549ec504f62aa55d9c2768dc3a6 (127.12.15.66:44897), af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085)
I20260806 03:51:42.333281 14546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c11cf60e3cdb4e8ca5c8de1eea9736f2" candidate_uuid: "92cec45e22e34a3eac634d313611b9a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" is_pre_election: true
I20260806 03:51:42.333262 14758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c11cf60e3cdb4e8ca5c8de1eea9736f2" candidate_uuid: "92cec45e22e34a3eac634d313611b9a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af649f700e8648caa447c0cdd08ab01e" is_pre_election: true
I20260806 03:51:42.333376 14758 raft_consensus.cc:2454] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92cec45e22e34a3eac634d313611b9a7 in term 0.
I20260806 03:51:42.333376 14546 raft_consensus.cc:2454] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 92cec45e22e34a3eac634d313611b9a7 in term 0.
I20260806 03:51:42.333544 14218 leader_election.cc:302] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92cec45e22e34a3eac634d313611b9a7, af649f700e8648caa447c0cdd08ab01e; no voters:
I20260806 03:51:42.333683 14895 raft_consensus.cc:2781] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:42.333739 14895 raft_consensus.cc:482] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:42.333766 14895 raft_consensus.cc:3037] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:42.334528 14895 raft_consensus.cc:504] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.334666 14895 leader_election.cc:288] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [CANDIDATE]: Term 1 election: Requested vote from peers 7f4ef549ec504f62aa55d9c2768dc3a6 (127.12.15.66:44897), af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085)
I20260806 03:51:42.334827 14758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c11cf60e3cdb4e8ca5c8de1eea9736f2" candidate_uuid: "92cec45e22e34a3eac634d313611b9a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "af649f700e8648caa447c0cdd08ab01e"
I20260806 03:51:42.334832 14546 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c11cf60e3cdb4e8ca5c8de1eea9736f2" candidate_uuid: "92cec45e22e34a3eac634d313611b9a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6"
I20260806 03:51:42.334897 14758 raft_consensus.cc:3037] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:42.334913 14546 raft_consensus.cc:3037] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:42.335551 14758 raft_consensus.cc:2454] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92cec45e22e34a3eac634d313611b9a7 in term 1.
I20260806 03:51:42.335563 14546 raft_consensus.cc:2454] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 92cec45e22e34a3eac634d313611b9a7 in term 1.
I20260806 03:51:42.335733 14218 leader_election.cc:302] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 92cec45e22e34a3eac634d313611b9a7, af649f700e8648caa447c0cdd08ab01e; no voters:
I20260806 03:51:42.335861 14895 raft_consensus.cc:2781] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:42.335979 14895 raft_consensus.cc:686] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [term 1 LEADER]: Becoming Leader. State: Replica: 92cec45e22e34a3eac634d313611b9a7, State: Running, Role: LEADER
I20260806 03:51:42.336036 14895 consensus_queue.cc:237] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } }
I20260806 03:51:42.336313 14934 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: TxnStatusTablet state changed. Reason: New leader 92cec45e22e34a3eac634d313611b9a7. Latest consensus state: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.336390 14934 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:51:42.336622 14937 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:51:42.336632 14081 catalog_manager.cc:5495] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 92cec45e22e34a3eac634d313611b9a7 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } health_report { overall_health: UNKNOWN } } }
I20260806 03:51:42.337584 14937 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:51:42.364585 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.361992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40061 (local address 127.0.0.1:39400)
0806 03:51:42.362065 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:42.362078 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:42.362150 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:42.362296 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:42.362296 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:42.362298 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:51:42.362348 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.362350 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.363230 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.363231 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.363814 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.363816 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.364251 (+ 435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.364252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.364293 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.364308 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:51:42.364452 (+ 144us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:51:42.364456 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:51:42.364486 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 03:51:42.364524 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 03:51:42.364595 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.362058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.12.15.65:40061)
0806 03:51:42.362145 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.362147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.362159 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:51:42.362216 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.362217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.362228 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.362290 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:42.362399 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.363157 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.363884 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.364191 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.364258 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.364372 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:42.364407 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:42.364453 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.364491 (+ 38us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.364512 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:51:42.365916 14758 raft_consensus.cc:1261] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Refusing update from remote peer 92cec45e22e34a3eac634d313611b9a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:42.365937 14546 raft_consensus.cc:1261] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Refusing update from remote peer 92cec45e22e34a3eac634d313611b9a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:42.366166 14895 consensus_queue.cc:1035] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:42.366302 14934 consensus_queue.cc:1035] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:42.366755 14865 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: TxnStatusTablet state changed. Reason: New leader 92cec45e22e34a3eac634d313611b9a7. Latest consensus state: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.366859 14865 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:42.366936 14863 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: TxnStatusTablet state changed. Reason: New leader 92cec45e22e34a3eac634d313611b9a7. Latest consensus state: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.367029 14863 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:42.367790 14895 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.367882 14943 mvcc.cc:205] Tried to move back new op lower bound from 7315408086489190400 to 7315408086368940032. Current Snapshot: MvccSnapshot[applied={T|T < 7315408086489190400}]
I20260806 03:51:42.367856 14934 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.367969 14934 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:51:42.367867 14895 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 92cec45e22e34a3eac634d313611b9a7: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:51:42.368186 14863 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.368259 14863 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P 7f4ef549ec504f62aa55d9c2768dc3a6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:42.368523 14865 tablet_replica.cc:439] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "92cec45e22e34a3eac634d313611b9a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 } } peers { permanent_uuid: "7f4ef549ec504f62aa55d9c2768dc3a6" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44897 } } peers { permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 } } }
I20260806 03:51:42.368603 14865 tablet_replica.cc:442] T c11cf60e3cdb4e8ca5c8de1eea9736f2 P af649f700e8648caa447c0cdd08ab01e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:42.369381 14947 mvcc.cc:205] Tried to move back new op lower bound from 7315408086489190400 to 7315408086368940032. Current Snapshot: MvccSnapshot[applied={T|T < 7315408086489190400 or (T in {7315408086489190400})}]
I20260806 03:51:42.372984 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.370320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55664 (local address 127.12.15.66:44897)
0806 03:51:42.370514 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.370515 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.370522 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:51:42.370566 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.370566 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.370587 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.370657 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:42.370797 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.371525 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.372197 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.372502 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.372564 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.372678 (+ 114us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:42.372807 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:42.372875 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.372890 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.372909 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":51,"threads_started":1}
I20260806 03:51:42.377727 14959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.375294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40061 (local address 127.12.15.66:33649)
0806 03:51:42.375435 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:42.375445 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:42.375492 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:42.375599 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:42.375600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:42.375601 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:51:42.375635 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.375636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.376424 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.376425 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.376983 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.376985 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.377409 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.377409 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.377445 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.377456 (+ 11us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:51:42.377592 (+ 136us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:51:42.377592 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 03:51:42.377620 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 03:51:42.377649 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260806 03:51:42.377719 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.375357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:33649 (local address 127.12.15.65:40061)
0806 03:51:42.375401 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.375403 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.375494 (+ 91us) server_negotiation.cc:407] Connection header received
0806 03:51:42.375530 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.375531 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.375540 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.375600 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:42.375676 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.376362 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.377049 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.377343 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.377404 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.377515 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:42.377546 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:42.377592 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.377624 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.377642 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:51:42.380112 14758 raft_consensus.cc:1261] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Refusing update from remote peer 7f4ef549ec504f62aa55d9c2768dc3a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:42.380136 14317 raft_consensus.cc:1261] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Refusing update from remote peer 7f4ef549ec504f62aa55d9c2768dc3a6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:42.380378 14914 consensus_queue.cc:1035] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "af649f700e8648caa447c0cdd08ab01e" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:42.380530 14914 consensus_queue.cc:1035] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92cec45e22e34a3eac634d313611b9a7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:42.388980 14956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.386285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:38939 (local address 127.12.15.66:44897)
0806 03:51:42.386322 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.386324 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.386405 (+ 81us) server_negotiation.cc:407] Connection header received
0806 03:51:42.386470 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.386470 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.386479 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.386514 (+ 35us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:42.386630 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.387060 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.388246 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.388539 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.388576 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.388795 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:42.388824 (+ 29us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:42.388857 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.388913 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.388930 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:51:42.389086 14969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.386083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:44897 (local address 127.12.15.65:38939)
0806 03:51:42.386342 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:42.386352 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:42.386416 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:42.386531 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:42.386531 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:42.386535 (+ 4us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:51:42.386580 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.386581 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.387122 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.387123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.388153 (+ 1030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:42.388157 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.388629 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.388630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:42.388695 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.388712 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:51:42.388863 (+ 151us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:51:42.388864 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:51:42.388910 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:51:42.388955 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":107,"threads_started":1}
I20260806 03:51:42.399111 14083 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:42.399240 14083 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57942:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:42.399857 14083 catalog_manager.cc:5782] T 00000000000000000000000000000000 P cc5257cbd0574721952a9ae069e310b4: Sending DeleteTablet for 3 replicas of tablet 7684dbacbe674348943080dcee42992d
I20260806 03:51:42.400126 14282 tablet_service.cc:1514] Processing DeleteTablet for tablet 7684dbacbe674348943080dcee42992d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:42 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:39382
I20260806 03:51:42.400265 14729 tablet_service.cc:1514] Processing DeleteTablet for tablet 7684dbacbe674348943080dcee42992d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:42 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:46276
I20260806 03:51:42.400453 14509 tablet_service.cc:1514] Processing DeleteTablet for tablet 7684dbacbe674348943080dcee42992d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:42 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:55660
I20260806 03:51:42.400471 14974 tablet_replica.cc:330] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: stopping tablet replica
I20260806 03:51:42.400588 14974 raft_consensus.cc:2229] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:42.400671 14974 raft_consensus.cc:2258] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:42.401005 14974 ts_tablet_manager.cc:1905] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:42.401437 14975 tablet_replica.cc:330] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: stopping tablet replica
I20260806 03:51:42.401511 14975 raft_consensus.cc:2229] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:42.401556 14975 raft_consensus.cc:2258] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:42.401684 14976 tablet_replica.cc:330] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: stopping tablet replica
I20260806 03:51:42.401753 14976 raft_consensus.cc:2229] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:51:42.401850 14976 raft_consensus.cc:2258] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:42.402594 14975 ts_tablet_manager.cc:1905] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:42.403187 14974 ts_tablet_manager.cc:1918] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:42.403299 14974 log.cc:1196] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/7684dbacbe674348943080dcee42992d
I20260806 03:51:42.403607 14974 ts_tablet_manager.cc:1939] T 7684dbacbe674348943080dcee42992d P 92cec45e22e34a3eac634d313611b9a7: Deleting consensus metadata
I20260806 03:51:42.403980 14062 catalog_manager.cc:4841] TS 92cec45e22e34a3eac634d313611b9a7 (127.12.15.65:40061): tablet 7684dbacbe674348943080dcee42992d (table test-table [id=5ffe7c36f9dd4a83bafb15c52bf0f257]) successfully deleted
I20260806 03:51:42.404191 14975 ts_tablet_manager.cc:1918] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:42.404253 14975 log.cc:1196] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/7684dbacbe674348943080dcee42992d
I20260806 03:51:42.404457 14975 ts_tablet_manager.cc:1939] T 7684dbacbe674348943080dcee42992d P af649f700e8648caa447c0cdd08ab01e: Deleting consensus metadata
I20260806 03:51:42.404760 14064 catalog_manager.cc:4841] TS af649f700e8648caa447c0cdd08ab01e (127.12.15.67:39085): tablet 7684dbacbe674348943080dcee42992d (table test-table [id=5ffe7c36f9dd4a83bafb15c52bf0f257]) successfully deleted
I20260806 03:51:42.404865 14976 ts_tablet_manager.cc:1905] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:42.406675 14976 ts_tablet_manager.cc:1918] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:42.406744 14976 log.cc:1196] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/7684dbacbe674348943080dcee42992d
I20260806 03:51:42.406965 14976 ts_tablet_manager.cc:1939] T 7684dbacbe674348943080dcee42992d P 7f4ef549ec504f62aa55d9c2768dc3a6: Deleting consensus metadata
I20260806 03:51:42.407300 14062 catalog_manager.cc:4841] TS 7f4ef549ec504f62aa55d9c2768dc3a6 (127.12.15.66:44897): tablet 7684dbacbe674348943080dcee42992d (table test-table [id=5ffe7c36f9dd4a83bafb15c52bf0f257]) successfully deleted
Aug 06 03:51:42 dist-test-slave-bqcl krb5kdc[14021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988301, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:51:42.416457 14851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.402144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39412 (local address 127.12.15.65:40061)
0806 03:51:42.402187 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.402189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.406301 (+ 4112us) server_negotiation.cc:407] Connection header received
0806 03:51:42.406415 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.406416 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.406427 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.406456 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:42.407491 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.408215 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.409332 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.409634 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.409674 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.413083 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:42.413093 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:42.413093 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:42.413101 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:42.415119 (+ 2018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:42.415546 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:42.415549 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:42.415550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:42.415623 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:42.415832 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:42.415833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:42.415833 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:42.416015 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:42.416180 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.416267 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.416356 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 03:51:42.416924 14385 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39412
I20260806 03:51:42.429906 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:42.418415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57970 (local address 127.12.15.126:35689)
0806 03:51:42.418467 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:42.418469 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:42.422304 (+ 3835us) server_negotiation.cc:407] Connection header received
0806 03:51:42.422366 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:42.422367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:42.422385 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:42.422447 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:42.423090 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.423906 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.424993 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:42.425331 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:42.425401 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:42.426954 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:42.426955 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:42.426956 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:42.426963 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:42.428631 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:42.429004 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:42.429005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:42.429006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:42.429070 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:42.429292 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:42.429293 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:42.429293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:42.429425 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:42.429507 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:42.429709 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 03:51:42.429804 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 03:51:42.430675 14083 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:57970
I20260806 03:51:42.431383 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14192
I20260806 03:51:42.447738 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14408
I20260806 03:51:42.457854 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14624
I20260806 03:51:42.466426 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14038
2026-08-06T03:51:42Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2910 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:42 dist-test-slave-bqcl krb5kdc[14985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:42 dist-test-slave-bqcl krb5kdc[14985](info): set up 2 sockets
Aug 06 03:51:42 dist-test-slave-bqcl krb5kdc[14985](info): commencing operation
krb5kdc: starting...
W20260806 03:51:43.600368 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.107s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:43 dist-test-slave-bqcl krb5kdc[14985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:43.712136 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:44397
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:44397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:43.791019 15003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:43.791208 15003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:43.791229 15003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:43.792320 15003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:43.792358 15003 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 03:51:43.792371 15003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:43.792392 15003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:43.792409 15003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:43.792431 15003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:43.793851 15003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:44397
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:44397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:43.794075 15003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:43.794293 15003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:43.798918 15011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:43.799068 15010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:43.798928 15016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:43.799598 15003 server_base.cc:1034] running on GCE node
I20260806 03:51:43.799862 15003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:43.800102 15003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:43.801232 15003 hybrid_clock.cc:648] HybridClock initialized: now 1785988303801224 us; error 29 us; skew 500 ppm
Aug 06 03:51:43 dist-test-slave-bqcl krb5kdc[14985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988303, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:43.803673 15003 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:43.804068 15003 webserver.cc:460] Webserver started at http://127.12.15.126:40883/ using document root <none> and password file <none>
I20260806 03:51:43.804257 15003 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:43.804301 15003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:43.804423 15003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:43.805423 15003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "a42095ed91624cb89513f0dd2eb0bd04"
format_stamp: "Formatted at 2026-08-06 03:51:43 on dist-test-slave-bqcl"
server_key: "e6536de0d5dc9a0b3d0bcaee7a0edf0e"
server_key_iv: "b77e2ab28cb1101dabc31a7226d40cfb"
server_key_version: "encryptionkey@0"
I20260806 03:51:43.805915 15003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "a42095ed91624cb89513f0dd2eb0bd04"
format_stamp: "Formatted at 2026-08-06 03:51:43 on dist-test-slave-bqcl"
server_key: "e6536de0d5dc9a0b3d0bcaee7a0edf0e"
server_key_iv: "b77e2ab28cb1101dabc31a7226d40cfb"
server_key_version: "encryptionkey@0"
I20260806 03:51:43.807179 15003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:43.807829 15031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:43.807976 15003 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:43.808055 15003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "a42095ed91624cb89513f0dd2eb0bd04"
format_stamp: "Formatted at 2026-08-06 03:51:43 on dist-test-slave-bqcl"
server_key: "e6536de0d5dc9a0b3d0bcaee7a0edf0e"
server_key_iv: "b77e2ab28cb1101dabc31a7226d40cfb"
server_key_version: "encryptionkey@0"
I20260806 03:51:43.808121 15003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:43.824883 15003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:43.827075 15003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:43.827210 15003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:43.831575 15003 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:44397
I20260806 03:51:43.831652 15131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:44397 every 8 connection(s)
I20260806 03:51:43.831946 15003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:43.832597 15132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:43.834877 15132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: Bootstrap starting.
I20260806 03:51:43.835412 15132 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:43.835636 15132 log.cc:824] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:43.836254 15132 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: No bootstrap required, opened a new log
I20260806 03:51:43.836483 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 15003
I20260806 03:51:43.836560 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:43.836731 12349 external_mini_cluster.cc:1428] Setting key cc7947cafff6b0211721e0c45024f524
I20260806 03:51:43.838805 15132 raft_consensus.cc:348] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42095ed91624cb89513f0dd2eb0bd04" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44397 } }
I20260806 03:51:43.838943 15132 raft_consensus.cc:374] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:43.838977 15132 raft_consensus.cc:729] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a42095ed91624cb89513f0dd2eb0bd04, State: Initialized, Role: FOLLOWER
I20260806 03:51:43.839138 15132 consensus_queue.cc:260] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42095ed91624cb89513f0dd2eb0bd04" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44397 } }
I20260806 03:51:43.839236 15132 raft_consensus.cc:388] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:43.839273 15132 raft_consensus.cc:482] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:43.839337 15132 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:43.839920 15132 raft_consensus.cc:504] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42095ed91624cb89513f0dd2eb0bd04" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44397 } }
I20260806 03:51:43.840008 15132 leader_election.cc:302] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a42095ed91624cb89513f0dd2eb0bd04; no voters:
I20260806 03:51:43.840169 15132 leader_election.cc:288] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:43.840214 15141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:43.840389 15141 raft_consensus.cc:686] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [term 1 LEADER]: Becoming Leader. State: Replica: a42095ed91624cb89513f0dd2eb0bd04, State: Running, Role: LEADER
I20260806 03:51:43.840415 15132 sys_catalog.cc:564] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:43.840499 15141 consensus_queue.cc:237] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42095ed91624cb89513f0dd2eb0bd04" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44397 } }
I20260806 03:51:43.840929 15144 sys_catalog.cc:455] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a42095ed91624cb89513f0dd2eb0bd04. Latest consensus state: current_term: 1 leader_uuid: "a42095ed91624cb89513f0dd2eb0bd04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42095ed91624cb89513f0dd2eb0bd04" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44397 } } }
I20260806 03:51:43.841018 15144 sys_catalog.cc:458] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:43.840935 15142 sys_catalog.cc:455] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a42095ed91624cb89513f0dd2eb0bd04" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a42095ed91624cb89513f0dd2eb0bd04" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44397 } } }
I20260806 03:51:43.841317 15142 sys_catalog.cc:458] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:43.841702 15151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:43.842237 15151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:43.844579 15151 catalog_manager.cc:1347] Generated new cluster ID: 3ce7ff76726643e69bad7a061ba2dc08
Aug 06 03:51:43 dist-test-slave-bqcl krb5kdc[14985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988303, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:43.844632 15151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:43.848755 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:43.837200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40502 (local address 127.12.15.126:44397)
0806 03:51:43.837357 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:43.837365 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:43.837390 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:51:43.838776 (+ 1386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:43.838782 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:43.838840 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:43.838963 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:43.839700 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:43.841033 (+ 1333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:43.842080 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:43.842564 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:43.842646 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:43.845117 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:43.845121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:43.845122 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:43.845134 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:43.847093 (+ 1959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:43.847439 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:43.847442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:43.847442 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:43.847609 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:43.847806 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:43.847807 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:43.847807 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:43.848119 (+ 312us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:43.848293 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:43.848365 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 03:51:43.848511 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":72,"threads_started":1}
I20260806 03:51:43.850078 15151 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:43.850642 15151 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:43.859427 15151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: Generated new TSK 0
I20260806 03:51:43.859575 15151 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:51:43.892799 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44397
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:43.974928 15176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:43.975121 15176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:43.975143 15176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:43.976246 15176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:43.976284 15176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:43.976303 15176 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 03:51:43.976313 15176 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:51:43.976332 15176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:43.977754 15176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:44397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:43.977970 15176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:43.978209 15176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:43.982872 15185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:43.982913 15183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:43.982973 15176 server_base.cc:1034] running on GCE node
W20260806 03:51:43.983132 15190 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:43.983323 15176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:43.983546 15176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:43.984695 15176 hybrid_clock.cc:648] HybridClock initialized: now 1785988303984677 us; error 32 us; skew 500 ppm
Aug 06 03:51:43 dist-test-slave-bqcl krb5kdc[14985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988303, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:43.987100 15176 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:51:43.987566 15176 webserver.cc:460] Webserver started at http://127.12.15.65:38771/ using document root <none> and password file <none>
I20260806 03:51:43.987771 15176 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:43.987819 15176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:43.987942 15176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:43.988914 15176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "0f28c66a48ee43ebaa21123c838a2a71"
format_stamp: "Formatted at 2026-08-06 03:51:43 on dist-test-slave-bqcl"
server_key: "fa1648c219017a1976f045c0001905d7"
server_key_iv: "8c8eb0fdc3250356fcc6cc8f98b469c1"
server_key_version: "encryptionkey@0"
I20260806 03:51:43.989248 15176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "0f28c66a48ee43ebaa21123c838a2a71"
format_stamp: "Formatted at 2026-08-06 03:51:43 on dist-test-slave-bqcl"
server_key: "fa1648c219017a1976f045c0001905d7"
server_key_iv: "8c8eb0fdc3250356fcc6cc8f98b469c1"
server_key_version: "encryptionkey@0"
I20260806 03:51:43.990478 15176 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:43.991117 15201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:43.991271 15176 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:43.991353 15176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "0f28c66a48ee43ebaa21123c838a2a71"
format_stamp: "Formatted at 2026-08-06 03:51:43 on dist-test-slave-bqcl"
server_key: "fa1648c219017a1976f045c0001905d7"
server_key_iv: "8c8eb0fdc3250356fcc6cc8f98b469c1"
server_key_version: "encryptionkey@0"
I20260806 03:51:43.991411 15176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:44.015273 15176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:44.017206 15176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:44.017339 15176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:44.017858 15176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:44.017897 15176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.017926 15176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:44.017943 15176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:51:44 dist-test-slave-bqcl krb5kdc[14985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988303, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:44.024741 15176 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:36607
I20260806 03:51:44.024847 15397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:36607 every 8 connection(s)
I20260806 03:51:44.025134 15176 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:51:44.026991 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.018300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:40125 (local address 127.12.15.126:44397)
0806 03:51:44.018344 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.018347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.018577 (+ 230us) server_negotiation.cc:407] Connection header received
0806 03:51:44.019078 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.019079 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.019096 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.019144 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:44.019957 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.020713 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.021746 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.021952 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.022010 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.024397 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:44.024398 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:44.024399 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:44.024407 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:44.025849 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.026279 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.026280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.026280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.026336 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.026505 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.026505 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.026506 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.026598 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:44.026640 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.026854 (+ 214us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.026926 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:44.027135 15216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.018307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44397 (local address 127.12.15.65:40125)
0806 03:51:44.018459 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.018496 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.018980 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.019234 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.019237 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.019559 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:44.019841 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.019847 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.020816 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.020817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.021650 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.021653 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.022043 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.022044 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.022217 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.022715 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:44.022727 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:44.024305 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:44.025914 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.025917 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.025919 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.026225 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.026379 (+ 154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.026379 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.026380 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.026471 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.026642 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:44.026645 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:44.026756 (+ 111us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.026832 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.026948 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":84,"threads_started":1}
I20260806 03:51:44.027606 15398 heartbeater.cc:347] Connected to a master server at 127.12.15.126:44397
I20260806 03:51:44.027688 15398 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:44.027863 15398 heartbeater.cc:511] Master 127.12.15.126:44397 requested a full tablet report, sending...
I20260806 03:51:44.027879 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 15176
I20260806 03:51:44.027971 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:44.028148 12349 external_mini_cluster.cc:1428] Setting key d03c62e8332b50335cda6fea2a332ffd
I20260806 03:51:44.028560 15057 ts_manager.cc:194] Registered new tserver with Master: 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607)
I20260806 03:51:44.029114 15057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:40125
I20260806 03:51:44.034211 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.028586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44397 (local address 127.12.15.65:39437)
0806 03:51:44.028700 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.028710 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.028757 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.028852 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.028852 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.029034 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:44.029101 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.029103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.029978 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.029978 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.030695 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.030698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.031105 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.031106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.031158 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.031522 (+ 364us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:44.031526 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:44.031792 (+ 266us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:44.033216 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.033217 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.033217 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.033369 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.033570 (+ 201us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.033571 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.033571 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.033623 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.033864 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:44.033864 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:44.033917 (+ 53us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.033985 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.034090 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":64,"threads_started":1}
I20260806 03:51:44.034219 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.028740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:39437 (local address 127.12.15.126:44397)
0806 03:51:44.028769 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.028771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.028777 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:51:44.028796 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.028796 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.028805 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.028849 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:44.029191 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.029876 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.030772 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.031040 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.031108 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.031870 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:44.031872 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:44.031872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:44.031879 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:44.033130 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.033437 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.033438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.033439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.033504 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.033683 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.033683 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.033683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.033809 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:44.033875 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.034020 (+ 145us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.034103 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:51:44.052649 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44397
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:44.131848 15410 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:44.132045 15410 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:44.132067 15410 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:44.133124 15410 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:44.133158 15410 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:44.133185 15410 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 03:51:44.133201 15410 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:51:44.133217 15410 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:44.134646 15410 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:44397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15410
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:44.134828 15410 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:44.135001 15410 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:44.139775 15420 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:44.139787 15424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:44.139789 15419 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:44.139938 15410 server_base.cc:1034] running on GCE node
I20260806 03:51:44.140139 15410 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:44.140352 15410 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:44.141501 15410 hybrid_clock.cc:648] HybridClock initialized: now 1785988304141511 us; error 49 us; skew 500 ppm
Aug 06 03:51:44 dist-test-slave-bqcl krb5kdc[14985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988304, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:44.143935 15410 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:51:44.144312 15410 webserver.cc:460] Webserver started at http://127.12.15.66:43327/ using document root <none> and password file <none>
I20260806 03:51:44.144516 15410 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:44.144565 15410 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:44.144702 15410 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:44.145823 15410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "4eea4280b68c4d51b66f7c7069a87ad2"
format_stamp: "Formatted at 2026-08-06 03:51:44 on dist-test-slave-bqcl"
server_key: "ff14717524e5e365fc9f2105891d30c0"
server_key_iv: "5070794fa98a53b8588ac360200122c3"
server_key_version: "encryptionkey@0"
I20260806 03:51:44.146229 15410 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "4eea4280b68c4d51b66f7c7069a87ad2"
format_stamp: "Formatted at 2026-08-06 03:51:44 on dist-test-slave-bqcl"
server_key: "ff14717524e5e365fc9f2105891d30c0"
server_key_iv: "5070794fa98a53b8588ac360200122c3"
server_key_version: "encryptionkey@0"
I20260806 03:51:44.147500 15410 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:44.148262 15431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.148435 15410 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:44.148522 15410 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "4eea4280b68c4d51b66f7c7069a87ad2"
format_stamp: "Formatted at 2026-08-06 03:51:44 on dist-test-slave-bqcl"
server_key: "ff14717524e5e365fc9f2105891d30c0"
server_key_iv: "5070794fa98a53b8588ac360200122c3"
server_key_version: "encryptionkey@0"
I20260806 03:51:44.148583 15410 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:44.166751 15410 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:44.168448 15410 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:44.168578 15410 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:44.169078 15410 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:44.169118 15410 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.169139 15410 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:44.169152 15410 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:51:44 dist-test-slave-bqcl krb5kdc[14985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988304, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:44.176083 15410 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:35425
I20260806 03:51:44.176424 15615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:35425 every 8 connection(s)
I20260806 03:51:44.176517 15410 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:51:44.177058 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 15410
I20260806 03:51:44.177155 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:44.177322 12349 external_mini_cluster.cc:1428] Setting key d53e5b5f0ecfc94fd6b50b2fa3371aea
I20260806 03:51:44.178747 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.169507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:36891 (local address 127.12.15.126:44397)
0806 03:51:44.169574 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.169576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.169807 (+ 231us) server_negotiation.cc:407] Connection header received
0806 03:51:44.170495 (+ 688us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.170496 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.170509 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.170574 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:44.171454 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.172004 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.172981 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.173184 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.173240 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.175852 (+ 2612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:44.175854 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:44.175854 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:44.175863 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:44.177361 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.177792 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.177793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.177794 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.177859 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.178155 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.178155 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.178156 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.178274 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:44.178327 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.178615 (+ 288us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.178683 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:51:44.178964 15442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.169484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44397 (local address 127.12.15.66:36891)
0806 03:51:44.169663 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.169714 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.170389 (+ 675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.170665 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.170670 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.171035 (+ 365us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:44.171349 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.171355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.172093 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.172094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.172900 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.172903 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.173256 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.173257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.173372 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.173838 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:44.173846 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:44.175755 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:44.177444 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.177452 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.177453 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.177724 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.177927 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.177928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.177928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.178066 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.178332 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:44.178338 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:44.178490 (+ 152us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.178603 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.178758 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260806 03:51:44.179383 15617 heartbeater.cc:347] Connected to a master server at 127.12.15.126:44397
I20260806 03:51:44.179479 15617 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:44.179677 15617 heartbeater.cc:511] Master 127.12.15.126:44397 requested a full tablet report, sending...
I20260806 03:51:44.180035 15058 ts_manager.cc:194] Registered new tserver with Master: 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425)
I20260806 03:51:44.180506 15058 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:36891
I20260806 03:51:44.186710 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.180765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44397 (local address 127.12.15.66:33235)
0806 03:51:44.181364 (+ 599us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.181375 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.181427 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.181536 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.181537 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.181685 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:44.181735 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.181736 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.182585 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.182586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.183085 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.183086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.183527 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.183528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.183576 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.183972 (+ 396us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:44.183975 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:44.184256 (+ 281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:44.185755 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.185756 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.185756 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.185910 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.186093 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.186094 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.186094 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.186170 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.186471 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:44.186471 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:44.186515 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.186565 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.186640 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":74,"threads_started":1}
I20260806 03:51:44.186764 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.180817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:33235 (local address 127.12.15.126:44397)
0806 03:51:44.180860 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.180862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.181438 (+ 576us) server_negotiation.cc:407] Connection header received
0806 03:51:44.181477 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.181478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.181489 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.181535 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:44.181793 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.182504 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.183161 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.183459 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.183526 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.184335 (+ 809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:44.184336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:44.184336 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:44.184343 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:44.185683 (+ 1340us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.185975 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.185976 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.185976 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.186035 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.186223 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.186224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.186224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.186408 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:44.186477 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.186608 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.186676 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:51:44.202932 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44397
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:44.283423 15628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:44.283612 15628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:44.283634 15628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:44.284688 15628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:44.284721 15628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:44.284740 15628 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260806 03:51:44.284754 15628 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:51:44.284770 15628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:44.286154 15628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:44397
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15628
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:44.286325 15628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:44.286500 15628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:44.291108 15639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:44.291101 15640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:44.291308 15628 server_base.cc:1034] running on GCE node
W20260806 03:51:44.291124 15644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:44.291512 15628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:44.291742 15628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:44.292883 15628 hybrid_clock.cc:648] HybridClock initialized: now 1785988304292867 us; error 28 us; skew 500 ppm
Aug 06 03:51:44 dist-test-slave-bqcl krb5kdc[14985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988304, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:44.295238 15628 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:51:44.295643 15628 webserver.cc:460] Webserver started at http://127.12.15.67:34849/ using document root <none> and password file <none>
I20260806 03:51:44.295861 15628 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:44.295908 15628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:44.296032 15628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:44.296977 15628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "01ea472f1d2940bdafb9b30024abfb15"
format_stamp: "Formatted at 2026-08-06 03:51:44 on dist-test-slave-bqcl"
server_key: "3614d0d611195df2bdb0adb11c8923a1"
server_key_iv: "def4fa67df780c769c45d64d3be684b2"
server_key_version: "encryptionkey@0"
I20260806 03:51:44.297331 15628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "01ea472f1d2940bdafb9b30024abfb15"
format_stamp: "Formatted at 2026-08-06 03:51:44 on dist-test-slave-bqcl"
server_key: "3614d0d611195df2bdb0adb11c8923a1"
server_key_iv: "def4fa67df780c769c45d64d3be684b2"
server_key_version: "encryptionkey@0"
I20260806 03:51:44.298717 15628 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:44.299341 15652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.299490 15628 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:44.299571 15628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "01ea472f1d2940bdafb9b30024abfb15"
format_stamp: "Formatted at 2026-08-06 03:51:44 on dist-test-slave-bqcl"
server_key: "3614d0d611195df2bdb0adb11c8923a1"
server_key_iv: "def4fa67df780c769c45d64d3be684b2"
server_key_version: "encryptionkey@0"
I20260806 03:51:44.299633 15628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:44.305620 15628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:44.307291 15628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:44.307416 15628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:44.307895 15628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:44.307936 15628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.307962 15628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:44.307986 15628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:51:44 dist-test-slave-bqcl krb5kdc[14985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988304, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:44.314996 15628 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:35091
I20260806 03:51:44.315065 15834 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:35091 every 8 connection(s)
I20260806 03:51:44.315461 15628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:51:44.317114 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.308259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:58813 (local address 127.12.15.126:44397)
0806 03:51:44.308326 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.308329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.308533 (+ 204us) server_negotiation.cc:407] Connection header received
0806 03:51:44.309114 (+ 581us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.309115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.309125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.309164 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:44.309967 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.310639 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.312195 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.312441 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.312503 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.314554 (+ 2051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:44.314555 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:44.314556 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:44.314562 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:44.316080 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.316403 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.316403 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.316404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.316452 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.316620 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.316620 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.316621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.316714 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:44.316757 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.316944 (+ 187us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.317021 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:51:44.317274 15663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.308243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44397 (local address 127.12.15.67:58813)
0806 03:51:44.308399 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.308446 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.309019 (+ 573us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.309248 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.309253 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.309592 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:44.309867 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.309874 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.310731 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.310733 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.312092 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.312095 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.312522 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.312524 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.312649 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.313062 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:44.313071 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:44.314442 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:44.316140 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.316144 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.316145 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.316346 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.316494 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.316494 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.316495 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.316580 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.316754 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:44.316757 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:44.316856 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.316929 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.317061 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":85,"threads_started":1}
I20260806 03:51:44.317436 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 15628
I20260806 03:51:44.317531 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:44.317708 12349 external_mini_cluster.cc:1428] Setting key 1c3efafc3b3377d8979a879b36a3098b
I20260806 03:51:44.318655 15835 heartbeater.cc:347] Connected to a master server at 127.12.15.126:44397
I20260806 03:51:44.318746 15835 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:44.318892 15835 heartbeater.cc:511] Master 127.12.15.126:44397 requested a full tablet report, sending...
I20260806 03:51:44.319226 15057 ts_manager.cc:194] Registered new tserver with Master: 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:44.319608 15057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:58813
I20260806 03:51:44.323148 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.318721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:60603 (local address 127.12.15.126:44397)
0806 03:51:44.318759 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.318761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.318881 (+ 120us) server_negotiation.cc:407] Connection header received
0806 03:51:44.318943 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.318944 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.318953 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.318997 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:44.319289 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.319797 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.320455 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.320627 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.320673 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.321348 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:44.321350 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:44.321350 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:44.321355 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:44.322388 (+ 1033us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.322659 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.322659 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.322660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.322709 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.322818 (+ 109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.322818 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.322818 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.322900 (+ 82us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:44.322938 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.323038 (+ 100us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.323090 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":19}
I20260806 03:51:44.323158 15843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.318676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44397 (local address 127.12.15.67:60603)
0806 03:51:44.318828 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.318837 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.318892 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.319000 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.319001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.319163 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:44.319226 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.319227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.319878 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.319878 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.320386 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.320387 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.320683 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.320684 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.320732 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.321026 (+ 294us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:44.321030 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:44.321283 (+ 253us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:44.322451 (+ 1168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.322452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.322452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.322605 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.322745 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.322745 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.322746 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.322787 (+ 41us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.322936 (+ 149us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:44.322936 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:44.322967 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.323022 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.323092 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":106,"threads_started":1}
I20260806 03:51:44.329284 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:51:44 dist-test-slave-bqcl krb5kdc[14985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:51:44 dist-test-slave-bqcl krb5kdc[14985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988304, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:44.351833 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.345841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40504 (local address 127.12.15.126:44397)
0806 03:51:44.345898 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.345900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.345909 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:44.345950 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.345951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.345960 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.345997 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:44.346534 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.347022 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.347687 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.347861 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.347902 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.349879 (+ 1977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:44.349881 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:44.349881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:44.349889 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:44.350982 (+ 1093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.351295 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.351296 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.351296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.351349 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.351475 (+ 126us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.351475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.351476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.351566 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:44.351635 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.351710 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.351768 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:51:44.353211 15057 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:51:44.354810 15057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:44.360652 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.356850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43826 (local address 127.12.15.66:35425)
0806 03:51:44.357156 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.357160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.357170 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:51:44.357208 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.357211 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.357225 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.357300 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:44.357471 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.358328 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.359623 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.360271 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.360332 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.360392 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.360529 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.360565 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":94,"threads_started":1}
I20260806 03:51:44.360644 15856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.356736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:35425 (local address 127.0.0.1:43826)
0806 03:51:44.356915 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.356958 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.357021 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.357313 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.357317 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.357319 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:44.357384 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.357385 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.358407 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.358408 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.359531 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.359533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.360342 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.360343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.360432 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.360447 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.360507 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.360558 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":67,"threads_started":1}
I20260806 03:51:44.360831 15862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.356935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36607 (local address 127.0.0.1:51714)
0806 03:51:44.357130 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.357139 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.357189 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.357564 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.357565 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.357567 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:44.357597 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.357598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.358873 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.358875 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.359680 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.359682 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.360616 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.360617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.360678 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.360684 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.360711 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.360753 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":44,"threads_started":1}
I20260806 03:51:44.361016 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.356778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:35091 (local address 127.0.0.1:36848)
0806 03:51:44.357021 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.357031 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.357076 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.357452 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.357453 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.357456 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:44.357526 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.357528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.358809 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.358810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.359854 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.359856 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.360828 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.360829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.360887 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.360890 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.360924 (+ 34us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.360958 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"mutex_wait_us":25,"thread_start_us":48,"threads_started":1}
I20260806 03:51:44.361239 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.357133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51714 (local address 127.12.15.65:36607)
0806 03:51:44.357290 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.357294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.357304 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:51:44.357337 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.357339 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.357353 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.357441 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:44.357679 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.358776 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.359751 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.360509 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.360993 (+ 484us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.361069 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.361097 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.361134 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":78,"threads_started":1}
I20260806 03:51:44.361380 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.356883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36848 (local address 127.12.15.67:35091)
0806 03:51:44.357303 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.357307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.357315 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:44.357351 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.357352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.357366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.357439 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:44.357604 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.358723 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.359938 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.360611 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.361166 (+ 555us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.361237 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.361261 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.361302 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":84,"threads_started":1}
I20260806 03:51:44.361420 15512 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6c88c7e7f442958401d6c40be1c864 (DEFAULT_TABLE table=test-table [id=4c29c92b0e2844599fcb86a18292ca68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:44.361773 15512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6c88c7e7f442958401d6c40be1c864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.361847 15288 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6c88c7e7f442958401d6c40be1c864 (DEFAULT_TABLE table=test-table [id=4c29c92b0e2844599fcb86a18292ca68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:44.361966 15730 tablet_service.cc:1467] Processing CreateTablet for tablet 6a6c88c7e7f442958401d6c40be1c864 (DEFAULT_TABLE table=test-table [id=4c29c92b0e2844599fcb86a18292ca68]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:44.362183 15288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6c88c7e7f442958401d6c40be1c864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.362314 15730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a6c88c7e7f442958401d6c40be1c864. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.364974 15864 tablet_bootstrap.cc:492] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Bootstrap starting.
I20260806 03:51:44.365263 15866 tablet_bootstrap.cc:492] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Bootstrap starting.
I20260806 03:51:44.365515 15867 tablet_bootstrap.cc:492] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Bootstrap starting.
I20260806 03:51:44.365936 15864 tablet_bootstrap.cc:654] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.365965 15866 tablet_bootstrap.cc:654] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.366034 15867 tablet_bootstrap.cc:654] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.366314 15866 log.cc:824] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:44.366328 15867 log.cc:824] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:44.366331 15864 log.cc:824] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:44.367069 15867 tablet_bootstrap.cc:492] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: No bootstrap required, opened a new log
I20260806 03:51:44.367069 15866 tablet_bootstrap.cc:492] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: No bootstrap required, opened a new log
I20260806 03:51:44.367143 15866 ts_tablet_manager.cc:1397] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:51:44.367143 15867 ts_tablet_manager.cc:1397] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:51:44.369021 15866 raft_consensus.cc:348] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.369021 15867 raft_consensus.cc:348] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.369194 15867 raft_consensus.cc:374] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.369194 15866 raft_consensus.cc:374] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.369238 15866 raft_consensus.cc:729] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eea4280b68c4d51b66f7c7069a87ad2, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.369238 15867 raft_consensus.cc:729] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f28c66a48ee43ebaa21123c838a2a71, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.369359 15867 consensus_queue.cc:260] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.369359 15866 consensus_queue.cc:260] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.369635 15867 ts_tablet_manager.cc:1428] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:51:44.369635 15866 ts_tablet_manager.cc:1428] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 03:51:44.369668 15398 heartbeater.cc:503] Master 127.12.15.126:44397 was elected leader, sending a full tablet report...
I20260806 03:51:44.369752 15617 heartbeater.cc:503] Master 127.12.15.126:44397 was elected leader, sending a full tablet report...
I20260806 03:51:44.370038 15864 tablet_bootstrap.cc:492] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: No bootstrap required, opened a new log
I20260806 03:51:44.370106 15864 ts_tablet_manager.cc:1397] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 03:51:44.371608 15864 raft_consensus.cc:348] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.371731 15864 raft_consensus.cc:374] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.371753 15864 raft_consensus.cc:729] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ea472f1d2940bdafb9b30024abfb15, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.371840 15864 consensus_queue.cc:260] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.372061 15864 ts_tablet_manager.cc:1428] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:44.372108 15835 heartbeater.cc:503] Master 127.12.15.126:44397 was elected leader, sending a full tablet report...
W20260806 03:51:44.427213 15619 tablet.cc:2369] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:51:44.525887 15400 tablet.cc:2369] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:51:44.562047 15872 raft_consensus.cc:482] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:44.562227 15872 raft_consensus.cc:504] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.562610 15872 leader_election.cc:288] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:44.565292 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.562870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:34451 (local address 127.12.15.66:35425)
0806 03:51:44.562931 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.562933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.563025 (+ 92us) server_negotiation.cc:407] Connection header received
0806 03:51:44.563090 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.563091 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.563099 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.563140 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:44.563267 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.563796 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.564650 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.564987 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.565045 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.565054 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.565190 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.565221 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:51:44.565306 15876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.562815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:35425 (local address 127.12.15.65:34451)
0806 03:51:44.562965 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.562974 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.563028 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.563148 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.563149 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.563151 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:44.563213 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.563215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.563865 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.563866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.564578 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.564579 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.565054 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.565054 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.565122 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.565125 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.565170 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.565206 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":1,"thread_start_us":89,"threads_started":1}
I20260806 03:51:44.565434 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.562876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:60759 (local address 127.12.15.67:35091)
0806 03:51:44.562922 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.562924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.562988 (+ 64us) server_negotiation.cc:407] Connection header received
0806 03:51:44.563045 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.563046 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.563055 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.563102 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:44.563240 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.563726 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.564679 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.565116 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.565189 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.565198 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.565333 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.565371 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:51:44.565447 15875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.562792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:35091 (local address 127.12.15.65:60759)
0806 03:51:44.562918 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.562927 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.562992 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.563109 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.563110 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.563112 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:44.563185 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.563187 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.563798 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.563800 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.564597 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.564599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.565192 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.565193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.565273 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.565276 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.565323 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.565362 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"mutex_wait_us":54,"thread_start_us":58,"threads_started":1}
I20260806 03:51:44.565654 15539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6c88c7e7f442958401d6c40be1c864" candidate_uuid: "0f28c66a48ee43ebaa21123c838a2a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" is_pre_election: true
I20260806 03:51:44.565819 15539 raft_consensus.cc:2454] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f28c66a48ee43ebaa21123c838a2a71 in term 0.
I20260806 03:51:44.565840 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6c88c7e7f442958401d6c40be1c864" candidate_uuid: "0f28c66a48ee43ebaa21123c838a2a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15" is_pre_election: true
I20260806 03:51:44.565986 15758 raft_consensus.cc:2454] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f28c66a48ee43ebaa21123c838a2a71 in term 0.
W20260806 03:51:44.566046 15836 tablet.cc:2369] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:51:44.566033 15205 leader_election.cc:302] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f28c66a48ee43ebaa21123c838a2a71, 4eea4280b68c4d51b66f7c7069a87ad2; no voters:
I20260806 03:51:44.566167 15872 raft_consensus.cc:2781] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:44.566229 15872 raft_consensus.cc:482] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:44.566250 15872 raft_consensus.cc:3037] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:44.566900 15872 raft_consensus.cc:504] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.567013 15872 leader_election.cc:288] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [CANDIDATE]: Term 1 election: Requested vote from peers 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:44.567152 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6c88c7e7f442958401d6c40be1c864" candidate_uuid: "0f28c66a48ee43ebaa21123c838a2a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15"
I20260806 03:51:44.567165 15539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a6c88c7e7f442958401d6c40be1c864" candidate_uuid: "0f28c66a48ee43ebaa21123c838a2a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eea4280b68c4d51b66f7c7069a87ad2"
I20260806 03:51:44.567235 15539 raft_consensus.cc:3037] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:44.567238 15758 raft_consensus.cc:3037] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:44.567898 15539 raft_consensus.cc:2454] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f28c66a48ee43ebaa21123c838a2a71 in term 1.
I20260806 03:51:44.567916 15758 raft_consensus.cc:2454] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f28c66a48ee43ebaa21123c838a2a71 in term 1.
I20260806 03:51:44.568063 15202 leader_election.cc:302] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ea472f1d2940bdafb9b30024abfb15, 0f28c66a48ee43ebaa21123c838a2a71; no voters:
I20260806 03:51:44.568156 15872 raft_consensus.cc:2781] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:44.568282 15872 raft_consensus.cc:686] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 LEADER]: Becoming Leader. State: Replica: 0f28c66a48ee43ebaa21123c838a2a71, State: Running, Role: LEADER
I20260806 03:51:44.568390 15872 consensus_queue.cc:237] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.569142 15058 catalog_manager.cc:5495] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "0f28c66a48ee43ebaa21123c838a2a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } health_report { overall_health: UNKNOWN } } }
Aug 06 03:51:44 dist-test-slave-bqcl krb5kdc[14985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988303, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:44.600548 15137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.594644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40506 (local address 127.12.15.126:44397)
0806 03:51:44.594716 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.594719 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.594808 (+ 89us) server_negotiation.cc:407] Connection header received
0806 03:51:44.594856 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.594857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.594866 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.594907 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:44.595330 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.595786 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.596420 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.596590 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.596639 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.598496 (+ 1857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:44.598497 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:44.598498 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:44.598505 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:44.599544 (+ 1039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.599864 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.599865 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.599866 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.599918 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:44.600073 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:44.600074 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:44.600074 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:44.600169 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:44.600224 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.600360 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.600450 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":50}
I20260806 03:51:44.600618 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.594591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44397 (local address 127.0.0.1:40506)
0806 03:51:44.594747 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.594756 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.594811 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.594921 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.594921 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.595214 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:44.595254 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.595256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.595849 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.595850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.596351 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.596352 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.596642 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.596643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.596687 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.597147 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:44.597152 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:44.598421 (+ 1269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:44.599610 (+ 1189us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.599617 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.599620 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.599804 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.599967 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:44.599967 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:44.599968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:44.600020 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:44.600220 (+ 200us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:44.600228 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:44.600276 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.600334 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.600513 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":80,"thread_start_us":79,"threads_started":1}
I20260806 03:51:44.602316 15058 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:40506:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 03:51:44.602485 15058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:44.603739 15288 tablet_service.cc:1467] Processing CreateTablet for tablet c16bdc2377c648cd9255fda13f6d31c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6aab115a7b274d3b850783d7961b6169]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:51:44.603778 15512 tablet_service.cc:1467] Processing CreateTablet for tablet c16bdc2377c648cd9255fda13f6d31c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6aab115a7b274d3b850783d7961b6169]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:51:44.603847 15730 tablet_service.cc:1467] Processing CreateTablet for tablet c16bdc2377c648cd9255fda13f6d31c7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=6aab115a7b274d3b850783d7961b6169]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:51:44.603924 15288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16bdc2377c648cd9255fda13f6d31c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.603924 15512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16bdc2377c648cd9255fda13f6d31c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.603996 15730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c16bdc2377c648cd9255fda13f6d31c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.605450 15867 tablet_bootstrap.cc:492] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: Bootstrap starting.
I20260806 03:51:44.605450 15866 tablet_bootstrap.cc:492] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: Bootstrap starting.
I20260806 03:51:44.605480 15864 tablet_bootstrap.cc:492] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: Bootstrap starting.
I20260806 03:51:44.605944 15864 tablet_bootstrap.cc:654] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.605983 15867 tablet_bootstrap.cc:654] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.606112 15866 tablet_bootstrap.cc:654] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.606714 15864 tablet_bootstrap.cc:492] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: No bootstrap required, opened a new log
I20260806 03:51:44.606765 15864 ts_tablet_manager.cc:1397] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:44.606887 15867 tablet_bootstrap.cc:492] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: No bootstrap required, opened a new log
I20260806 03:51:44.606940 15867 ts_tablet_manager.cc:1397] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:51:44.606915 15864 raft_consensus.cc:348] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.606984 15864 raft_consensus.cc:374] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.607003 15864 raft_consensus.cc:729] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ea472f1d2940bdafb9b30024abfb15, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.607048 15864 consensus_queue.cc:260] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.607131 15864 ts_tablet_manager.cc:1428] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.607087 15867 raft_consensus.cc:348] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.607160 15867 raft_consensus.cc:374] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.607188 15867 raft_consensus.cc:729] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f28c66a48ee43ebaa21123c838a2a71, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.607192 15873 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.607224 15867 consensus_queue.cc:260] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.607264 15873 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:44.607321 15867 ts_tablet_manager.cc:1428] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.607362 15872 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.607427 15872 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:44.607486 15866 tablet_bootstrap.cc:492] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: No bootstrap required, opened a new log
I20260806 03:51:44.607527 15866 ts_tablet_manager.cc:1397] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:51:44.607681 15866 raft_consensus.cc:348] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.607753 15866 raft_consensus.cc:374] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.607774 15866 raft_consensus.cc:729] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eea4280b68c4d51b66f7c7069a87ad2, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.607810 15866 consensus_queue.cc:260] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.607889 15866 ts_tablet_manager.cc:1428] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.607980 15871 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.608085 15871 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:44.759590 15871 raft_consensus.cc:482] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:44.759696 15871 raft_consensus.cc:504] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.760046 15871 leader_election.cc:288] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:44.763865 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.760313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:36543 (local address 127.12.15.65:36607)
0806 03:51:44.760360 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.760362 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.760463 (+ 101us) server_negotiation.cc:407] Connection header received
0806 03:51:44.760525 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.760526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.760535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.760596 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:44.760758 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.761515 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.762960 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.763504 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.763579 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.763588 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.763750 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.763790 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:51:44.763865 15861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.760270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:50613 (local address 127.12.15.67:35091)
0806 03:51:44.760316 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.760318 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.760449 (+ 131us) server_negotiation.cc:407] Connection header received
0806 03:51:44.760525 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.760526 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.760535 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.760596 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:44.760793 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.761509 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.762960 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.763504 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.763579 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.763588 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.763750 (+ 162us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.763790 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:44.763899 15895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.760208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36607 (local address 127.12.15.66:36543)
0806 03:51:44.760375 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.760386 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.760456 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.760612 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.760612 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.760616 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:44.760690 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.760693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.761589 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.761590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.762884 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.762888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.763587 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.763588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.763678 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.763683 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.763739 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.763792 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":27,"thread_start_us":106,"threads_started":1}
I20260806 03:51:44.763907 15894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.760208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:35091 (local address 127.12.15.66:50613)
0806 03:51:44.760390 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.760399 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.760456 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.760612 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.760612 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.760615 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:44.760731 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.760732 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.761588 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.761589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.762884 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.762888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.763585 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.763586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.763692 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.763695 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.763741 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.763792 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":101,"spinlock_wait_cycles":896,"thread_start_us":107,"threads_started":1}
I20260806 03:51:44.764228 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16bdc2377c648cd9255fda13f6d31c7" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f28c66a48ee43ebaa21123c838a2a71" is_pre_election: true
I20260806 03:51:44.764351 15325 raft_consensus.cc:2454] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 0.
I20260806 03:51:44.764336 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16bdc2377c648cd9255fda13f6d31c7" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15" is_pre_election: true
I20260806 03:51:44.764411 15758 raft_consensus.cc:2454] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 0.
I20260806 03:51:44.764536 15436 leader_election.cc:302] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f28c66a48ee43ebaa21123c838a2a71, 4eea4280b68c4d51b66f7c7069a87ad2; no voters:
I20260806 03:51:44.764674 15871 raft_consensus.cc:2781] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:44.764717 15871 raft_consensus.cc:482] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:44.764736 15871 raft_consensus.cc:3037] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:44.765430 15871 raft_consensus.cc:504] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.765547 15871 leader_election.cc:288] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 election: Requested vote from peers 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:44.765697 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16bdc2377c648cd9255fda13f6d31c7" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f28c66a48ee43ebaa21123c838a2a71"
I20260806 03:51:44.765774 15325 raft_consensus.cc:3037] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:44.765751 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c16bdc2377c648cd9255fda13f6d31c7" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15"
I20260806 03:51:44.765810 15758 raft_consensus.cc:3037] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:44.766445 15758 raft_consensus.cc:2454] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 1.
I20260806 03:51:44.766461 15325 raft_consensus.cc:2454] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 1.
I20260806 03:51:44.766613 15436 leader_election.cc:302] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f28c66a48ee43ebaa21123c838a2a71, 4eea4280b68c4d51b66f7c7069a87ad2; no voters:
I20260806 03:51:44.766713 15871 raft_consensus.cc:2781] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:44.766815 15871 raft_consensus.cc:686] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 LEADER]: Becoming Leader. State: Replica: 4eea4280b68c4d51b66f7c7069a87ad2, State: Running, Role: LEADER
I20260806 03:51:44.766896 15871 consensus_queue.cc:237] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.767272 15896 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: TxnStatusTablet state changed. Reason: New leader 4eea4280b68c4d51b66f7c7069a87ad2. Latest consensus state: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.767386 15896 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:51:44.767575 15899 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:51:44.767702 15899 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:51:44.767638 15057 catalog_manager.cc:5495] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66). New cstate: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } health_report { overall_health: UNKNOWN } } }
I20260806 03:51:44.787109 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.784297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.12.15.66:35425)
0806 03:51:44.784334 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.784336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.784371 (+ 35us) server_negotiation.cc:407] Connection header received
0806 03:51:44.784465 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.784466 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.784480 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.784535 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:44.784662 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.785241 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.786406 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.786585 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.786638 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.786829 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:44.786921 (+ 92us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:44.786972 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.787040 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.787058 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:51:44.787170 15888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.784242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:35425 (local address 127.0.0.1:43834)
0806 03:51:44.784327 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.784338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.784390 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.784555 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.784556 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.784557 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:51:44.784606 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.784609 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.785359 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.785361 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.786326 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.786330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.786654 (+ 324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.786655 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.786726 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.786744 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:51:44.786973 (+ 229us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:51:44.786979 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.787037 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.787086 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260806 03:51:44.788908 15325 raft_consensus.cc:1261] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Refusing update from remote peer 4eea4280b68c4d51b66f7c7069a87ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:44.789155 15758 raft_consensus.cc:1261] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Refusing update from remote peer 4eea4280b68c4d51b66f7c7069a87ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:44.789204 15871 consensus_queue.cc:1035] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:44.789427 15871 consensus_queue.cc:1035] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:44.790706 15873 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: TxnStatusTablet state changed. Reason: New leader 4eea4280b68c4d51b66f7c7069a87ad2. Latest consensus state: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.790795 15873 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:44.791230 15871 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.791200 15872 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: TxnStatusTablet state changed. Reason: New leader 4eea4280b68c4d51b66f7c7069a87ad2. Latest consensus state: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.791285 15871 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:51:44.791286 15872 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:44.791337 15872 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.791371 15872 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 0f28c66a48ee43ebaa21123c838a2a71: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:44.791519 15903 mvcc.cc:205] Tried to move back new op lower bound from 7315408096413093888 to 7315408096325767168. Current Snapshot: MvccSnapshot[applied={T|T < 7315408096413093888}]
I20260806 03:51:44.791821 15896 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.791888 15896 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 4eea4280b68c4d51b66f7c7069a87ad2: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:51:44.792274 15873 tablet_replica.cc:439] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } } }
I20260806 03:51:44.792356 15873 tablet_replica.cc:442] T c16bdc2377c648cd9255fda13f6d31c7 P 01ea472f1d2940bdafb9b30024abfb15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:51:44.792586 15902 mvcc.cc:205] Tried to move back new op lower bound from 7315408096413093888 to 7315408096325767168. Current Snapshot: MvccSnapshot[applied={T|T < 7315408096413093888}]
I20260806 03:51:44.793304 15907 mvcc.cc:205] Tried to move back new op lower bound from 7315408096413093888 to 7315408096325767168. Current Snapshot: MvccSnapshot[applied={T|T < 7315408096413093888}]
I20260806 03:51:44.796887 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.794416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51716 (local address 127.12.15.65:36607)
0806 03:51:44.794451 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.794453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.794466 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:51:44.794541 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.794541 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.794556 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.794610 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:44.794716 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.795196 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.796274 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.796450 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.796496 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.796639 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:44.796724 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:44.796774 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.796821 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.796838 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:51:44.801278 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.799157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:39933 (local address 127.12.15.66:35425)
0806 03:51:44.799197 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.799199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.799307 (+ 108us) server_negotiation.cc:407] Connection header received
0806 03:51:44.799352 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.799352 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.799361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.799407 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:44.799528 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.799962 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.800681 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.800854 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.800904 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.801076 (+ 172us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:44.801107 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:44.801148 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.801195 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.801215 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:51:44.801316 15917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.799091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:35425 (local address 127.12.15.65:39933)
0806 03:51:44.799237 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.799248 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.799307 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.799414 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.799415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.799416 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:51:44.799477 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.799479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.800029 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.800030 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.800609 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.800611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.800910 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.800911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.800984 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.800998 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:51:44.801154 (+ 156us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:51:44.801155 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.801194 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.801236 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":81,"threads_started":1}
I20260806 03:51:44.803349 15758 raft_consensus.cc:1261] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Refusing update from remote peer 0f28c66a48ee43ebaa21123c838a2a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:44.803407 15539 raft_consensus.cc:1261] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Refusing update from remote peer 0f28c66a48ee43ebaa21123c838a2a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:44.803633 15877 consensus_queue.cc:1035] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:44.803763 15872 consensus_queue.cc:1035] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:44.812111 15863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.809387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:36255 (local address 127.12.15.65:36607)
0806 03:51:44.809425 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:44.809426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:44.809538 (+ 112us) server_negotiation.cc:407] Connection header received
0806 03:51:44.809587 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:44.809587 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:44.809596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:44.809659 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:44.809767 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.810495 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.811498 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.811704 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.811756 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.811907 (+ 151us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:44.811939 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:44.811981 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:44.812029 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 03:51:44.812047 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:51:44.812139 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:44.809323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36607 (local address 127.12.15.66:36255)
0806 03:51:44.809476 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:44.809485 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:44.809541 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:44.809667 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:44.809667 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:44.809678 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:51:44.809719 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.809720 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.810577 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.810578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.811395 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:44.811398 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:44.811766 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:44.811767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:44.811830 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:44.811844 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:51:44.811987 (+ 143us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:51:44.811989 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:51:44.812028 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 03:51:44.812067 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260806 03:51:44.821821 15058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:44.821923 15058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:44.822628 15058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: Sending DeleteTablet for 3 replicas of tablet 6a6c88c7e7f442958401d6c40be1c864
I20260806 03:51:44.822921 15730 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6c88c7e7f442958401d6c40be1c864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:44 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:36848
I20260806 03:51:44.823082 15512 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6c88c7e7f442958401d6c40be1c864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:44 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:43826
I20260806 03:51:44.823082 15288 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a6c88c7e7f442958401d6c40be1c864 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:44 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:51714
I20260806 03:51:44.823318 15931 tablet_replica.cc:330] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: stopping tablet replica
I20260806 03:51:44.823434 15058 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40504:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:51:44.823571 15058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:44.823833 15932 tablet_replica.cc:330] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: stopping tablet replica
I20260806 03:51:44.824011 15932 raft_consensus.cc:2229] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:44.824147 15931 raft_consensus.cc:2229] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:51:44.824285 15931 raft_consensus.cc:2258] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:44.824748 15730 tablet_service.cc:1467] Processing CreateTablet for tablet f7d1dfd1151f4c4687a876406fc740cd (DEFAULT_TABLE table=test-table [id=c0d48f16dc0349d6b64baa6c35c2a0a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:44.824867 15730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d1dfd1151f4c4687a876406fc740cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.826236 15864 tablet_bootstrap.cc:492] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: Bootstrap starting.
I20260806 03:51:44.826382 15512 tablet_service.cc:1467] Processing CreateTablet for tablet f7d1dfd1151f4c4687a876406fc740cd (DEFAULT_TABLE table=test-table [id=c0d48f16dc0349d6b64baa6c35c2a0a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:44.826611 15512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d1dfd1151f4c4687a876406fc740cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.826633 15864 tablet_bootstrap.cc:654] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.827814 15866 tablet_bootstrap.cc:492] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: Bootstrap starting.
I20260806 03:51:44.824486 15932 raft_consensus.cc:2258] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:44.826454 15288 tablet_service.cc:1467] Processing CreateTablet for tablet f7d1dfd1151f4c4687a876406fc740cd (DEFAULT_TABLE table=test-table [id=c0d48f16dc0349d6b64baa6c35c2a0a3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:44.828050 15288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f7d1dfd1151f4c4687a876406fc740cd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:44.828301 15866 tablet_bootstrap.cc:654] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.828413 15929 tablet_replica.cc:330] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: stopping tablet replica
I20260806 03:51:44.828516 15929 raft_consensus.cc:2229] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:44.828560 15929 raft_consensus.cc:2258] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:44.829383 15867 tablet_bootstrap.cc:492] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: Bootstrap starting.
I20260806 03:51:44.829416 15866 tablet_bootstrap.cc:492] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: No bootstrap required, opened a new log
I20260806 03:51:44.829459 15866 ts_tablet_manager.cc:1397] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:51:44.829579 15866 raft_consensus.cc:348] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.829638 15866 raft_consensus.cc:374] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.829656 15866 raft_consensus.cc:729] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eea4280b68c4d51b66f7c7069a87ad2, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.829704 15866 consensus_queue.cc:260] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.829790 15866 ts_tablet_manager.cc:1428] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.829799 15867 tablet_bootstrap.cc:654] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:44.830067 15932 ts_tablet_manager.cc:1905] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:44.830663 15931 ts_tablet_manager.cc:1905] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:44.832188 15931 ts_tablet_manager.cc:1918] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:44.832262 15931 log.cc:1196] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/6a6c88c7e7f442958401d6c40be1c864
I20260806 03:51:44.833556 15932 ts_tablet_manager.cc:1918] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:44.833627 15932 log.cc:1196] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/6a6c88c7e7f442958401d6c40be1c864
I20260806 03:51:44.833875 15932 ts_tablet_manager.cc:1939] T 6a6c88c7e7f442958401d6c40be1c864 P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting consensus metadata
I20260806 03:51:44.834275 15040 catalog_manager.cc:4841] TS 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425): tablet 6a6c88c7e7f442958401d6c40be1c864 (table test-table [id=4c29c92b0e2844599fcb86a18292ca68]) successfully deleted
I20260806 03:51:44.834590 15931 ts_tablet_manager.cc:1939] T 6a6c88c7e7f442958401d6c40be1c864 P 0f28c66a48ee43ebaa21123c838a2a71: Deleting consensus metadata
I20260806 03:51:44.834672 15929 ts_tablet_manager.cc:1905] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:44.834959 15040 catalog_manager.cc:4841] TS 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607): tablet 6a6c88c7e7f442958401d6c40be1c864 (table test-table [id=4c29c92b0e2844599fcb86a18292ca68]) successfully deleted
I20260806 03:51:44.835170 15867 tablet_bootstrap.cc:492] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: No bootstrap required, opened a new log
I20260806 03:51:44.835213 15867 ts_tablet_manager.cc:1397] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260806 03:51:44.835330 15867 raft_consensus.cc:348] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.835387 15867 raft_consensus.cc:374] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.835399 15867 raft_consensus.cc:729] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f28c66a48ee43ebaa21123c838a2a71, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.835435 15867 consensus_queue.cc:260] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.835525 15867 ts_tablet_manager.cc:1428] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:44.836577 15929 ts_tablet_manager.cc:1918] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:44.836635 15929 log.cc:1196] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/6a6c88c7e7f442958401d6c40be1c864
I20260806 03:51:44.836790 15929 ts_tablet_manager.cc:1939] T 6a6c88c7e7f442958401d6c40be1c864 P 01ea472f1d2940bdafb9b30024abfb15: Deleting consensus metadata
I20260806 03:51:44.837253 15033 catalog_manager.cc:4841] TS 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091): tablet 6a6c88c7e7f442958401d6c40be1c864 (table test-table [id=4c29c92b0e2844599fcb86a18292ca68]) successfully deleted
I20260806 03:51:44.840765 15864 tablet_bootstrap.cc:492] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: No bootstrap required, opened a new log
I20260806 03:51:44.840822 15864 ts_tablet_manager.cc:1397] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: Time spent bootstrapping tablet: real 0.015s user 0.000s sys 0.001s
I20260806 03:51:44.840969 15864 raft_consensus.cc:348] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.841027 15864 raft_consensus.cc:374] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:44.841045 15864 raft_consensus.cc:729] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ea472f1d2940bdafb9b30024abfb15, State: Initialized, Role: FOLLOWER
I20260806 03:51:44.841089 15864 consensus_queue.cc:260] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:44.841176 15864 ts_tablet_manager.cc:1428] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:45.108567 15918 raft_consensus.cc:482] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:45.108685 15918 raft_consensus.cc:504] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:45.108872 15918 leader_election.cc:288] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:45.109062 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d1dfd1151f4c4687a876406fc740cd" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f28c66a48ee43ebaa21123c838a2a71" is_pre_election: true
I20260806 03:51:45.109072 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d1dfd1151f4c4687a876406fc740cd" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15" is_pre_election: true
I20260806 03:51:45.109176 15758 raft_consensus.cc:2454] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 0.
I20260806 03:51:45.109176 15325 raft_consensus.cc:2454] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 0.
I20260806 03:51:45.109333 15432 leader_election.cc:302] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ea472f1d2940bdafb9b30024abfb15, 4eea4280b68c4d51b66f7c7069a87ad2; no voters:
I20260806 03:51:45.109443 15918 raft_consensus.cc:2781] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:45.109514 15918 raft_consensus.cc:482] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:45.109534 15918 raft_consensus.cc:3037] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:45.110308 15918 raft_consensus.cc:504] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:45.110432 15918 leader_election.cc:288] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 election: Requested vote from peers 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:45.110606 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d1dfd1151f4c4687a876406fc740cd" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f28c66a48ee43ebaa21123c838a2a71"
I20260806 03:51:45.110630 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f7d1dfd1151f4c4687a876406fc740cd" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15"
I20260806 03:51:45.110684 15325 raft_consensus.cc:3037] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:45.110707 15758 raft_consensus.cc:3037] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:45.111449 15325 raft_consensus.cc:2454] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 1.
I20260806 03:51:45.111449 15758 raft_consensus.cc:2454] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 1.
I20260806 03:51:45.111641 15432 leader_election.cc:302] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ea472f1d2940bdafb9b30024abfb15, 4eea4280b68c4d51b66f7c7069a87ad2; no voters:
I20260806 03:51:45.111761 15918 raft_consensus.cc:2781] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:45.111814 15918 raft_consensus.cc:686] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 LEADER]: Becoming Leader. State: Replica: 4eea4280b68c4d51b66f7c7069a87ad2, State: Running, Role: LEADER
I20260806 03:51:45.111876 15918 consensus_queue.cc:237] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:45.112437 15058 catalog_manager.cc:5495] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66). New cstate: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } health_report { overall_health: UNKNOWN } } }
I20260806 03:51:45.182281 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:45.180100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43844 (local address 127.12.15.66:35425)
0806 03:51:45.180141 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:45.180143 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:45.180149 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:51:45.180174 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:45.180175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:45.180186 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:45.180246 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:45.180337 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:45.180986 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:45.181727 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:45.181925 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:45.181973 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:45.182053 (+ 80us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:45.182081 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:45.182111 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:45.182186 (+ 75us) server_negotiation.cc:299] Negotiation successful
0806 03:51:45.182207 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:51:45.185998 15923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:45.184065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:35425 (local address 127.12.15.66:33923)
0806 03:51:45.184115 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:45.184124 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:45.184160 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:45.184275 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:45.184275 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:45.184276 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:51:45.184312 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:45.184313 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:45.184864 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:45.184865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:45.185381 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:45.185382 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:45.185678 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:45.185678 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:45.185724 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:45.185728 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:51:45.185858 (+ 130us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:51:45.185859 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:51:45.185895 (+ 36us) client_negotiation.cc:240] Negotiation successful
0806 03:51:45.185935 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:51:45.185998 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:45.184120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:33923 (local address 127.12.15.66:35425)
0806 03:51:45.184161 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:45.184163 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:45.184170 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:51:45.184221 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:45.184221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:45.184232 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:45.184274 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:45.184364 (+ 90us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:45.184798 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:45.185451 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:45.185621 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:45.185671 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:45.185782 (+ 111us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:45.185810 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:45.185850 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:45.185909 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 03:51:45.185924 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":27}
I20260806 03:51:45.187671 15325 raft_consensus.cc:1261] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Refusing update from remote peer 4eea4280b68c4d51b66f7c7069a87ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:45.187691 15758 raft_consensus.cc:1261] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Refusing update from remote peer 4eea4280b68c4d51b66f7c7069a87ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:45.187867 15896 consensus_queue.cc:1035] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:45.187973 15918 consensus_queue.cc:1035] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 03:51:45.192251 15902 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 03:51:46.710498 15058 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:46.710597 15058 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40504:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:46.711402 15058 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: Sending DeleteTablet for 3 replicas of tablet f7d1dfd1151f4c4687a876406fc740cd
I20260806 03:51:46.711627 15730 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d1dfd1151f4c4687a876406fc740cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:46 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:36848
I20260806 03:51:46.711625 15288 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d1dfd1151f4c4687a876406fc740cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:46 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:51714
I20260806 03:51:46.711647 15512 tablet_service.cc:1514] Processing DeleteTablet for tablet f7d1dfd1151f4c4687a876406fc740cd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:46 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:43826
I20260806 03:51:46.711862 15983 tablet_replica.cc:330] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: stopping tablet replica
I20260806 03:51:46.711918 15983 raft_consensus.cc:2229] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:51:46.711997 15983 raft_consensus.cc:2258] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:46.712225 15983 ts_tablet_manager.cc:1905] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:46.712420 15982 tablet_replica.cc:330] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: stopping tablet replica
I20260806 03:51:46.712484 15982 raft_consensus.cc:2229] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:46.712554 15982 raft_consensus.cc:2258] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:46.712663 15981 tablet_replica.cc:330] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: stopping tablet replica
I20260806 03:51:46.712834 15982 ts_tablet_manager.cc:1905] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:46.713963 15983 ts_tablet_manager.cc:1918] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:51:46.714087 15983 log.cc:1196] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/f7d1dfd1151f4c4687a876406fc740cd
I20260806 03:51:46.714411 15983 ts_tablet_manager.cc:1939] T f7d1dfd1151f4c4687a876406fc740cd P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting consensus metadata
I20260806 03:51:46.714717 15982 ts_tablet_manager.cc:1918] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:51:46.714783 15982 log.cc:1196] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/f7d1dfd1151f4c4687a876406fc740cd
I20260806 03:51:46.714759 15040 catalog_manager.cc:4841] TS 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425): tablet f7d1dfd1151f4c4687a876406fc740cd (table test-table [id=c0d48f16dc0349d6b64baa6c35c2a0a3]) successfully deleted
I20260806 03:51:46.714995 15982 ts_tablet_manager.cc:1939] T f7d1dfd1151f4c4687a876406fc740cd P 01ea472f1d2940bdafb9b30024abfb15: Deleting consensus metadata
I20260806 03:51:46.715227 15981 raft_consensus.cc:2229] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:46.715288 15981 raft_consensus.cc:2258] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:46.715276 15033 catalog_manager.cc:4841] TS 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091): tablet f7d1dfd1151f4c4687a876406fc740cd (table test-table [id=c0d48f16dc0349d6b64baa6c35c2a0a3]) successfully deleted
I20260806 03:51:46.715495 15981 ts_tablet_manager.cc:1905] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:46.716651 15981 ts_tablet_manager.cc:1918] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:51:46.716706 15981 log.cc:1196] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/f7d1dfd1151f4c4687a876406fc740cd
I20260806 03:51:46.716879 15981 ts_tablet_manager.cc:1939] T f7d1dfd1151f4c4687a876406fc740cd P 0f28c66a48ee43ebaa21123c838a2a71: Deleting consensus metadata
I20260806 03:51:46.717140 15040 catalog_manager.cc:4841] TS 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607): tablet f7d1dfd1151f4c4687a876406fc740cd (table test-table [id=c0d48f16dc0349d6b64baa6c35c2a0a3]) successfully deleted
Aug 06 03:51:46 dist-test-slave-bqcl krb5kdc[14985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988306, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:51:46 dist-test-slave-bqcl krb5kdc[14985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988306, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:46.736783 15998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:46.728149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53926 (local address 127.12.15.126:44397)
0806 03:51:46.728289 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:46.728291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:46.728299 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:46.728335 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:46.728336 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:46.728346 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:46.728394 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:46.728907 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:46.729410 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:46.730088 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:46.730318 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:46.730367 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:46.734801 (+ 4434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:46.734802 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:46.734802 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:46.734809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:46.735962 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:46.736259 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:46.736260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:46.736260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:46.736307 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:46.736432 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:46.736432 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:46.736433 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:46.736525 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:46.736594 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:46.736661 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 03:51:46.736718 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":78,"threads_started":1}
I20260806 03:51:46.737978 15058 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:51:46.738116 15058 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:46.739305 15512 tablet_service.cc:1467] Processing CreateTablet for tablet 2755b5cd708a4ccdbc4b5f5edfa00369 (DEFAULT_TABLE table=test-table [id=e2c70167ba9e4a0485fc480c76edee5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:46.739322 15288 tablet_service.cc:1467] Processing CreateTablet for tablet 2755b5cd708a4ccdbc4b5f5edfa00369 (DEFAULT_TABLE table=test-table [id=e2c70167ba9e4a0485fc480c76edee5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:46.739396 15730 tablet_service.cc:1467] Processing CreateTablet for tablet 2755b5cd708a4ccdbc4b5f5edfa00369 (DEFAULT_TABLE table=test-table [id=e2c70167ba9e4a0485fc480c76edee5d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:46.739432 15512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2755b5cd708a4ccdbc4b5f5edfa00369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:46.739444 15288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2755b5cd708a4ccdbc4b5f5edfa00369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:46.739499 15730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2755b5cd708a4ccdbc4b5f5edfa00369. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:46.740361 15999 tablet_bootstrap.cc:492] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: Bootstrap starting.
I20260806 03:51:46.740749 15999 tablet_bootstrap.cc:654] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:46.740882 16003 tablet_bootstrap.cc:492] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: Bootstrap starting.
I20260806 03:51:46.740981 16002 tablet_bootstrap.cc:492] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: Bootstrap starting.
I20260806 03:51:46.741331 16003 tablet_bootstrap.cc:654] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:46.741353 16002 tablet_bootstrap.cc:654] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:46.741425 15999 tablet_bootstrap.cc:492] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: No bootstrap required, opened a new log
I20260806 03:51:46.741483 15999 ts_tablet_manager.cc:1397] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:46.741601 15999 raft_consensus.cc:348] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.741677 15999 raft_consensus.cc:374] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:46.741698 15999 raft_consensus.cc:729] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f28c66a48ee43ebaa21123c838a2a71, State: Initialized, Role: FOLLOWER
I20260806 03:51:46.741751 15999 consensus_queue.cc:260] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.741918 15999 ts_tablet_manager.cc:1428] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:46.741931 16002 tablet_bootstrap.cc:492] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: No bootstrap required, opened a new log
I20260806 03:51:46.741969 16002 ts_tablet_manager.cc:1397] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:46.742151 16002 raft_consensus.cc:348] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.742221 16002 raft_consensus.cc:374] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:46.742249 16002 raft_consensus.cc:729] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ea472f1d2940bdafb9b30024abfb15, State: Initialized, Role: FOLLOWER
I20260806 03:51:46.742295 16002 consensus_queue.cc:260] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.742455 16002 ts_tablet_manager.cc:1428] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:46.742748 16003 tablet_bootstrap.cc:492] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: No bootstrap required, opened a new log
I20260806 03:51:46.742791 16003 ts_tablet_manager.cc:1397] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:51:46.742888 16003 raft_consensus.cc:348] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.742947 16003 raft_consensus.cc:374] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:46.742967 16003 raft_consensus.cc:729] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eea4280b68c4d51b66f7c7069a87ad2, State: Initialized, Role: FOLLOWER
I20260806 03:51:46.743013 16003 consensus_queue.cc:260] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.743105 16003 ts_tablet_manager.cc:1428] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:46.981551 15975 raft_consensus.cc:482] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:46.981655 15975 raft_consensus.cc:504] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.981837 15975 leader_election.cc:288] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:46.982046 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2755b5cd708a4ccdbc4b5f5edfa00369" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f28c66a48ee43ebaa21123c838a2a71" is_pre_election: true
I20260806 03:51:46.982077 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2755b5cd708a4ccdbc4b5f5edfa00369" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15" is_pre_election: true
I20260806 03:51:46.982164 15325 raft_consensus.cc:2454] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 0.
I20260806 03:51:46.982228 15758 raft_consensus.cc:2454] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 0.
I20260806 03:51:46.982347 15436 leader_election.cc:302] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f28c66a48ee43ebaa21123c838a2a71, 4eea4280b68c4d51b66f7c7069a87ad2; no voters:
I20260806 03:51:46.982453 15975 raft_consensus.cc:2781] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:46.982497 15975 raft_consensus.cc:482] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:46.982513 15975 raft_consensus.cc:3037] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:46.983243 15975 raft_consensus.cc:504] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.983367 15975 leader_election.cc:288] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 election: Requested vote from peers 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:46.983510 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2755b5cd708a4ccdbc4b5f5edfa00369" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15"
I20260806 03:51:46.983521 15325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2755b5cd708a4ccdbc4b5f5edfa00369" candidate_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f28c66a48ee43ebaa21123c838a2a71"
I20260806 03:51:46.983585 15325 raft_consensus.cc:3037] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:46.983585 15758 raft_consensus.cc:3037] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:46.984287 15758 raft_consensus.cc:2454] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 1.
I20260806 03:51:46.984309 15325 raft_consensus.cc:2454] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4eea4280b68c4d51b66f7c7069a87ad2 in term 1.
I20260806 03:51:46.984447 15436 leader_election.cc:302] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f28c66a48ee43ebaa21123c838a2a71, 4eea4280b68c4d51b66f7c7069a87ad2; no voters:
I20260806 03:51:46.984572 15975 raft_consensus.cc:2781] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:46.984635 15975 raft_consensus.cc:686] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 LEADER]: Becoming Leader. State: Replica: 4eea4280b68c4d51b66f7c7069a87ad2, State: Running, Role: LEADER
I20260806 03:51:46.984688 15975 consensus_queue.cc:237] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:46.985307 15058 catalog_manager.cc:5495] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66). New cstate: current_term: 1 leader_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } health_report { overall_health: UNKNOWN } } }
I20260806 03:51:47.025921 16020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:47.023528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58624 (local address 127.12.15.66:35425)
0806 03:51:47.023702 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:47.023704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:47.023712 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:47.023733 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:47.023734 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:47.023745 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:47.023807 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:47.023955 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:47.024428 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:47.025220 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:47.025525 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:47.025596 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:47.025700 (+ 104us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:47.025735 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:47.025781 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:47.025818 (+ 37us) server_negotiation.cc:299] Negotiation successful
0806 03:51:47.025840 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":90,"threads_started":1}
I20260806 03:51:47.028656 15758 raft_consensus.cc:1261] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Refusing update from remote peer 4eea4280b68c4d51b66f7c7069a87ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:47.028656 15325 raft_consensus.cc:1261] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Refusing update from remote peer 4eea4280b68c4d51b66f7c7069a87ad2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:47.028867 15975 consensus_queue.cc:1035] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:47.028976 15871 consensus_queue.cc:1035] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:47.038537 15056 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:47.038630 15056 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:47.039222 15056 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: Sending DeleteTablet for 3 replicas of tablet 2755b5cd708a4ccdbc4b5f5edfa00369
I20260806 03:51:47.039429 15288 tablet_service.cc:1514] Processing DeleteTablet for tablet 2755b5cd708a4ccdbc4b5f5edfa00369 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:47 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:51714
I20260806 03:51:47.039467 15730 tablet_service.cc:1514] Processing DeleteTablet for tablet 2755b5cd708a4ccdbc4b5f5edfa00369 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:47 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:36848
I20260806 03:51:47.039482 15512 tablet_service.cc:1514] Processing DeleteTablet for tablet 2755b5cd708a4ccdbc4b5f5edfa00369 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:47 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:43826
I20260806 03:51:47.039559 15983 tablet_replica.cc:330] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: stopping tablet replica
I20260806 03:51:47.039559 15981 tablet_replica.cc:330] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: stopping tablet replica
I20260806 03:51:47.039609 15983 raft_consensus.cc:2229] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:51:47.039618 15981 raft_consensus.cc:2229] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:47.039655 15981 raft_consensus.cc:2258] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:47.039660 15983 raft_consensus.cc:2258] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:47.039726 15982 tablet_replica.cc:330] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: stopping tablet replica
I20260806 03:51:47.039781 15982 raft_consensus.cc:2229] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:47.039840 15982 raft_consensus.cc:2258] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:47.039768 15056 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53926:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:51:47.039915 15056 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:47.040246 15981 ts_tablet_manager.cc:1905] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:47.040387 15983 ts_tablet_manager.cc:1905] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:47.041731 15982 ts_tablet_manager.cc:1905] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:47.041831 15981 ts_tablet_manager.cc:1918] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:47.041877 15981 log.cc:1196] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/2755b5cd708a4ccdbc4b5f5edfa00369
I20260806 03:51:47.042030 15983 ts_tablet_manager.cc:1918] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:47.042059 15981 ts_tablet_manager.cc:1939] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 0f28c66a48ee43ebaa21123c838a2a71: Deleting consensus metadata
I20260806 03:51:47.042083 15983 log.cc:1196] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/2755b5cd708a4ccdbc4b5f5edfa00369
I20260806 03:51:47.042296 15983 ts_tablet_manager.cc:1939] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting consensus metadata
I20260806 03:51:47.042387 15040 catalog_manager.cc:4841] TS 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607): tablet 2755b5cd708a4ccdbc4b5f5edfa00369 (table test-table [id=e2c70167ba9e4a0485fc480c76edee5d]) successfully deleted
I20260806 03:51:47.042577 15288 tablet_service.cc:1467] Processing CreateTablet for tablet 0650df12f0444640aefb45502b71379a (DEFAULT_TABLE table=test-table [id=823874803cdb4b63bacf204c87d4072f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:47.042618 15730 tablet_service.cc:1467] Processing CreateTablet for tablet 0650df12f0444640aefb45502b71379a (DEFAULT_TABLE table=test-table [id=823874803cdb4b63bacf204c87d4072f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:47.042673 15288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0650df12f0444640aefb45502b71379a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:47.042766 15040 catalog_manager.cc:4841] TS 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425): tablet 2755b5cd708a4ccdbc4b5f5edfa00369 (table test-table [id=e2c70167ba9e4a0485fc480c76edee5d]) successfully deleted
I20260806 03:51:47.042775 15512 tablet_service.cc:1467] Processing CreateTablet for tablet 0650df12f0444640aefb45502b71379a (DEFAULT_TABLE table=test-table [id=823874803cdb4b63bacf204c87d4072f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:47.042858 15512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0650df12f0444640aefb45502b71379a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:47.043296 15730 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0650df12f0444640aefb45502b71379a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:47.043417 15982 ts_tablet_manager.cc:1918] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:51:47.043462 15982 log.cc:1196] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/2755b5cd708a4ccdbc4b5f5edfa00369
I20260806 03:51:47.043643 15982 ts_tablet_manager.cc:1939] T 2755b5cd708a4ccdbc4b5f5edfa00369 P 01ea472f1d2940bdafb9b30024abfb15: Deleting consensus metadata
I20260806 03:51:47.043959 15999 tablet_bootstrap.cc:492] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71: Bootstrap starting.
I20260806 03:51:47.043978 15033 catalog_manager.cc:4841] TS 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091): tablet 2755b5cd708a4ccdbc4b5f5edfa00369 (table test-table [id=e2c70167ba9e4a0485fc480c76edee5d]) successfully deleted
I20260806 03:51:47.044251 15999 tablet_bootstrap.cc:654] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:47.044410 16002 tablet_bootstrap.cc:492] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: Bootstrap starting.
I20260806 03:51:47.044421 16003 tablet_bootstrap.cc:492] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: Bootstrap starting.
I20260806 03:51:47.044715 16003 tablet_bootstrap.cc:654] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:47.044885 15999 tablet_bootstrap.cc:492] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71: No bootstrap required, opened a new log
I20260806 03:51:47.044910 16002 tablet_bootstrap.cc:654] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:47.044929 15999 ts_tablet_manager.cc:1397] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:47.045030 15999 raft_consensus.cc:348] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.045087 15999 raft_consensus.cc:374] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:47.045109 15999 raft_consensus.cc:729] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f28c66a48ee43ebaa21123c838a2a71, State: Initialized, Role: FOLLOWER
I20260806 03:51:47.045162 16003 tablet_bootstrap.cc:492] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: No bootstrap required, opened a new log
I20260806 03:51:47.045158 15999 consensus_queue.cc:260] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.045205 16003 ts_tablet_manager.cc:1397] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 03:51:47.045258 15999 ts_tablet_manager.cc:1428] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:47.045322 16003 raft_consensus.cc:348] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.045375 16003 raft_consensus.cc:374] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:47.045387 16003 raft_consensus.cc:729] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4eea4280b68c4d51b66f7c7069a87ad2, State: Initialized, Role: FOLLOWER
I20260806 03:51:47.045420 16003 consensus_queue.cc:260] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.045490 16003 ts_tablet_manager.cc:1428] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:47.045624 16002 tablet_bootstrap.cc:492] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: No bootstrap required, opened a new log
I20260806 03:51:47.045670 16002 ts_tablet_manager.cc:1397] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:47.045795 16002 raft_consensus.cc:348] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.045855 16002 raft_consensus.cc:374] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:47.045874 16002 raft_consensus.cc:729] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01ea472f1d2940bdafb9b30024abfb15, State: Initialized, Role: FOLLOWER
I20260806 03:51:47.045915 16002 consensus_queue.cc:260] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.045993 16002 ts_tablet_manager.cc:1428] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:47.105599 16012 raft_consensus.cc:482] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:47.105700 16012 raft_consensus.cc:504] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.105859 16012 leader_election.cc:288] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:47.106051 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0650df12f0444640aefb45502b71379a" candidate_uuid: "0f28c66a48ee43ebaa21123c838a2a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15" is_pre_election: true
I20260806 03:51:47.106082 15539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0650df12f0444640aefb45502b71379a" candidate_uuid: "0f28c66a48ee43ebaa21123c838a2a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" is_pre_election: true
I20260806 03:51:47.106170 15758 raft_consensus.cc:2454] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f28c66a48ee43ebaa21123c838a2a71 in term 0.
I20260806 03:51:47.106222 15539 raft_consensus.cc:2454] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f28c66a48ee43ebaa21123c838a2a71 in term 0.
I20260806 03:51:47.106325 15202 leader_election.cc:302] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ea472f1d2940bdafb9b30024abfb15, 0f28c66a48ee43ebaa21123c838a2a71; no voters:
I20260806 03:51:47.106436 16012 raft_consensus.cc:2781] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:47.106487 16012 raft_consensus.cc:482] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:47.106506 16012 raft_consensus.cc:3037] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:47.107220 16012 raft_consensus.cc:504] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.107337 16012 leader_election.cc:288] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [CANDIDATE]: Term 1 election: Requested vote from peers 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425), 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091)
I20260806 03:51:47.107474 15539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0650df12f0444640aefb45502b71379a" candidate_uuid: "0f28c66a48ee43ebaa21123c838a2a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4eea4280b68c4d51b66f7c7069a87ad2"
I20260806 03:51:47.107475 15758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0650df12f0444640aefb45502b71379a" candidate_uuid: "0f28c66a48ee43ebaa21123c838a2a71" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "01ea472f1d2940bdafb9b30024abfb15"
I20260806 03:51:47.107542 15539 raft_consensus.cc:3037] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:47.107542 15758 raft_consensus.cc:3037] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:47.108285 15539 raft_consensus.cc:2454] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f28c66a48ee43ebaa21123c838a2a71 in term 1.
I20260806 03:51:47.108285 15758 raft_consensus.cc:2454] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f28c66a48ee43ebaa21123c838a2a71 in term 1.
I20260806 03:51:47.108467 15202 leader_election.cc:302] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01ea472f1d2940bdafb9b30024abfb15, 0f28c66a48ee43ebaa21123c838a2a71; no voters:
I20260806 03:51:47.108551 16012 raft_consensus.cc:2781] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:47.108677 16012 raft_consensus.cc:686] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [term 1 LEADER]: Becoming Leader. State: Replica: 0f28c66a48ee43ebaa21123c838a2a71, State: Running, Role: LEADER
I20260806 03:51:47.108723 16012 consensus_queue.cc:237] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } }
I20260806 03:51:47.109336 15056 catalog_manager.cc:5495] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "0f28c66a48ee43ebaa21123c838a2a71" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f28c66a48ee43ebaa21123c838a2a71" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36607 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 } health_report { overall_health: UNKNOWN } } }
I20260806 03:51:47.120206 16032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:47.117932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40520 (local address 127.12.15.65:36607)
0806 03:51:47.118099 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:47.118101 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:47.118108 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:51:47.118142 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:47.118143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:47.118154 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:47.118208 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:47.118326 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:47.118852 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:47.119577 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:47.119795 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:47.119848 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:47.119979 (+ 131us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:47.120014 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:47.120052 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:47.120100 (+ 48us) server_negotiation.cc:299] Negotiation successful
0806 03:51:47.120123 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":93,"threads_started":1}
I20260806 03:51:47.122188 15539 raft_consensus.cc:1261] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Refusing update from remote peer 0f28c66a48ee43ebaa21123c838a2a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:47.122236 15758 raft_consensus.cc:1261] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Refusing update from remote peer 0f28c66a48ee43ebaa21123c838a2a71: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:47.122375 16029 consensus_queue.cc:1035] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4eea4280b68c4d51b66f7c7069a87ad2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 35425 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:47.122478 16012 consensus_queue.cc:1035] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71 [LEADER]: Connected to new peer: Peer: permanent_uuid: "01ea472f1d2940bdafb9b30024abfb15" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35091 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260806 03:51:47.126924 15955 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260806 03:51:48.642760 15056 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:48.642880 15056 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53926:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:48.643590 15056 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a42095ed91624cb89513f0dd2eb0bd04: Sending DeleteTablet for 3 replicas of tablet 0650df12f0444640aefb45502b71379a
I20260806 03:51:48.643843 15512 tablet_service.cc:1514] Processing DeleteTablet for tablet 0650df12f0444640aefb45502b71379a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:48 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:43826
I20260806 03:51:48.643860 15288 tablet_service.cc:1514] Processing DeleteTablet for tablet 0650df12f0444640aefb45502b71379a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:48 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:51714
I20260806 03:51:48.643859 15730 tablet_service.cc:1514] Processing DeleteTablet for tablet 0650df12f0444640aefb45502b71379a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:48 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:36848
I20260806 03:51:48.644026 16080 tablet_replica.cc:330] T 0650df12f0444640aefb45502b71379a P 0f28c66a48ee43ebaa21123c838a2a71: stopping tablet replica
I20260806 03:51:48.644021 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 15176
I20260806 03:51:48.644060 16079 tablet_replica.cc:330] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: stopping tablet replica
I20260806 03:51:48.651031 16079 raft_consensus.cc:2229] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:48.651155 16079 raft_consensus.cc:2258] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:48.651489 16079 ts_tablet_manager.cc:1905] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:48.653151 16081 tablet_replica.cc:330] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: stopping tablet replica
I20260806 03:51:48.653225 16081 raft_consensus.cc:2229] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:48.653271 16081 raft_consensus.cc:2258] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:48.653378 16079 ts_tablet_manager.cc:1918] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:51:48.653447 16079 log.cc:1196] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/0650df12f0444640aefb45502b71379a
I20260806 03:51:48.653641 16079 ts_tablet_manager.cc:1939] T 0650df12f0444640aefb45502b71379a P 4eea4280b68c4d51b66f7c7069a87ad2: Deleting consensus metadata
I20260806 03:51:48.654196 16081 ts_tablet_manager.cc:1905] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:48.655048 15040 catalog_manager.cc:4841] TS 4eea4280b68c4d51b66f7c7069a87ad2 (127.12.15.66:35425): tablet 0650df12f0444640aefb45502b71379a (table test-table [id=823874803cdb4b63bacf204c87d4072f]) successfully deleted
I20260806 03:51:48.655730 16081 ts_tablet_manager.cc:1918] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260806 03:51:48.655776 16081 log.cc:1196] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/0650df12f0444640aefb45502b71379a
I20260806 03:51:48.655941 16081 ts_tablet_manager.cc:1939] T 0650df12f0444640aefb45502b71379a P 01ea472f1d2940bdafb9b30024abfb15: Deleting consensus metadata
I20260806 03:51:48.656282 15033 catalog_manager.cc:4841] TS 01ea472f1d2940bdafb9b30024abfb15 (127.12.15.67:35091): tablet 0650df12f0444640aefb45502b71379a (table test-table [id=823874803cdb4b63bacf204c87d4072f]) successfully deleted
W20260806 03:51:48.662228 15040 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:36607 (error 108)
I20260806 03:51:48.662490 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 15410
I20260806 03:51:48.670456 16083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:48.662676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36607 (local address 127.0.0.1:40536)
0806 03:51:48.670321 (+ 7645us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:48.670399 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:36607: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":7609,"thread_start_us":62,"threads_started":1}
W20260806 03:51:48.670620 15040 catalog_manager.cc:4584] TS 0f28c66a48ee43ebaa21123c838a2a71 (127.12.15.65:36607): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0650df12f0444640aefb45502b71379a: Network error: Client connection negotiation failed: client connection to 127.12.15.65:36607: connect: Connection refused (error 111)
I20260806 03:51:48.675094 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 15628
I20260806 03:51:48.687418 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 15003
2026-08-06T03:51:48Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6222 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:48 dist-test-slave-bqcl krb5kdc[16089](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:48 dist-test-slave-bqcl krb5kdc[16089](info): set up 2 sockets
Aug 06 03:51:48 dist-test-slave-bqcl krb5kdc[16089](info): commencing operation
krb5kdc: starting...
W20260806 03:51:49.828783 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:49 dist-test-slave-bqcl krb5kdc[16089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:49.945952 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:40917
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:40917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:50.027292 16112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:50.027493 16112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:50.027513 16112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:50.028713 16112 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:50.028757 16112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:50.028770 16112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:50.028781 16112 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:50.028801 16112 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:50.030313 16112 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:40917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:40917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:50.030543 16112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:50.030769 16112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:50.036914 16121 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:50.036974 16125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:50.036993 16120 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:50.037034 16112 server_base.cc:1034] running on GCE node
I20260806 03:51:50.037379 16112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:50.037688 16112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:50.038823 16112 hybrid_clock.cc:648] HybridClock initialized: now 1785988310038806 us; error 33 us; skew 500 ppm
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988310, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:50.041260 16112 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:50.041684 16112 webserver.cc:460] Webserver started at http://127.12.15.126:43373/ using document root <none> and password file <none>
I20260806 03:51:50.041882 16112 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:50.041921 16112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:50.042012 16112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:50.043057 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "8b88334505aa41f18b24c2a1d9b60ee4"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "f8bf38a84c3f9404b1f47eec70a1aafa"
server_key_iv: "6cd0fc72432a02d65187e3eaa19e8a75"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.043359 16112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "8b88334505aa41f18b24c2a1d9b60ee4"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "f8bf38a84c3f9404b1f47eec70a1aafa"
server_key_iv: "6cd0fc72432a02d65187e3eaa19e8a75"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.044574 16112 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:50.045219 16135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.045403 16112 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:50.045490 16112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "8b88334505aa41f18b24c2a1d9b60ee4"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "f8bf38a84c3f9404b1f47eec70a1aafa"
server_key_iv: "6cd0fc72432a02d65187e3eaa19e8a75"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.045544 16112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:50.063642 16112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:50.065740 16112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:50.065850 16112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:50.069502 16112 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:40917
I20260806 03:51:50.069554 16219 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:40917 every 8 connection(s)
I20260806 03:51:50.069855 16112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:50.070063 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 16112
I20260806 03:51:50.070171 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:50.070362 12349 external_mini_cluster.cc:1428] Setting key d29512826615be2e9bde54c65a8b80d0
I20260806 03:51:50.070848 16220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:50.073902 16220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4: Bootstrap starting.
I20260806 03:51:50.074543 16220 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:50.074848 16220 log.cc:824] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:50.075552 16220 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4: No bootstrap required, opened a new log
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988309, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:50.077335 16220 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b88334505aa41f18b24c2a1d9b60ee4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40917 } }
I20260806 03:51:50.077504 16220 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:50.077543 16220 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b88334505aa41f18b24c2a1d9b60ee4, State: Initialized, Role: FOLLOWER
I20260806 03:51:50.077641 16220 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b88334505aa41f18b24c2a1d9b60ee4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40917 } }
I20260806 03:51:50.077708 16220 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:50.077729 16220 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:50.077759 16220 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:50.078315 16220 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b88334505aa41f18b24c2a1d9b60ee4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40917 } }
I20260806 03:51:50.078402 16220 leader_election.cc:302] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8b88334505aa41f18b24c2a1d9b60ee4; no voters:
I20260806 03:51:50.078547 16220 leader_election.cc:288] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:50.078591 16227 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:50.078769 16227 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [term 1 LEADER]: Becoming Leader. State: Replica: 8b88334505aa41f18b24c2a1d9b60ee4, State: Running, Role: LEADER
I20260806 03:51:50.078819 16220 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:50.078884 16227 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b88334505aa41f18b24c2a1d9b60ee4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40917 } }
I20260806 03:51:50.079277 16227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8b88334505aa41f18b24c2a1d9b60ee4. Latest consensus state: current_term: 1 leader_uuid: "8b88334505aa41f18b24c2a1d9b60ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b88334505aa41f18b24c2a1d9b60ee4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40917 } } }
I20260806 03:51:50.079388 16227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:50.080058 16229 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8b88334505aa41f18b24c2a1d9b60ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8b88334505aa41f18b24c2a1d9b60ee4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40917 } } }
I20260806 03:51:50.080139 16229 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4 [sys.catalog]: This master's current role is: LEADER
W20260806 03:51:50.080765 16257 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:50.080859 16257 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:50.080996 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:50.070846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49628 (local address 127.12.15.126:40917)
0806 03:51:50.071026 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:50.071035 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:50.071067 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:51:50.071560 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:50.071570 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:50.071631 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:50.071780 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:50.072323 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.073635 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.074425 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.075014 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.075119 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:50.077240 (+ 2121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:50.077244 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:50.077244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:50.077256 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:50.079037 (+ 1781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:50.079406 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:50.079409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:50.079409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:50.079512 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:50.079802 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:50.079803 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:50.079804 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:50.080123 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:50.080303 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:50.080716 (+ 413us) server_negotiation.cc:299] Negotiation successful
0806 03:51:50.080837 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":97,"threads_started":1}
I20260806 03:51:50.080976 16239 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:50.081188 16239 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:50.082685 16239 catalog_manager.cc:1347] Generated new cluster ID: 669a5170765e41ba8fe591528f7c14a8
I20260806 03:51:50.082741 16239 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:50.092664 16239 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:50.093346 16239 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:50.103356 16239 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8b88334505aa41f18b24c2a1d9b60ee4: Generated new TSK 0
I20260806 03:51:50.103521 16239 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:51:50.133075 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40917
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:50.214993 16264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:50.215199 16264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:50.215222 16264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:50.216368 16264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:50.216414 16264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:50.216444 16264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:50.217880 16264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:50.218158 16264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:50.218379 16264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:50.223151 16277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:50.223162 16272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:50.223157 16273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:50.223489 16264 server_base.cc:1034] running on GCE node
I20260806 03:51:50.223732 16264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:50.223951 16264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:50.225085 16264 hybrid_clock.cc:648] HybridClock initialized: now 1785988310225071 us; error 32 us; skew 500 ppm
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988310, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:50.227612 16264 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:51:50.228018 16264 webserver.cc:460] Webserver started at http://127.12.15.65:33401/ using document root <none> and password file <none>
I20260806 03:51:50.228205 16264 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:50.228245 16264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:50.228335 16264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:50.229336 16264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "97345799da8a4b71a5bcecff8960aaa6"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "1e295e1910da65e393b243d8f50950ea"
server_key_iv: "dbc5e1a47c1f920548af6532224907fd"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.229640 16264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "97345799da8a4b71a5bcecff8960aaa6"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "1e295e1910da65e393b243d8f50950ea"
server_key_iv: "dbc5e1a47c1f920548af6532224907fd"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.231076 16264 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:50.231698 16298 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.231901 16264 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:50.231993 16264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "97345799da8a4b71a5bcecff8960aaa6"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "1e295e1910da65e393b243d8f50950ea"
server_key_iv: "dbc5e1a47c1f920548af6532224907fd"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.232074 16264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:50.246563 16264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:50.248610 16264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:50.248739 16264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:50.248941 16264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:50.249291 16264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:50.249327 16264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.249356 16264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:50.249379 16264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.254841 16264 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:37981
I20260806 03:51:50.254876 16482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:37981 every 8 connection(s)
I20260806 03:51:50.255185 16264 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:51:50.257849 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 16264
I20260806 03:51:50.257934 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:50.258083 12349 external_mini_cluster.cc:1428] Setting key 340374333af04fc9b99869f2df237ac0
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988310, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:50.265048 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:50.255834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:56067 (local address 127.12.15.126:40917)
0806 03:51:50.255906 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:50.255909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:50.256109 (+ 200us) server_negotiation.cc:407] Connection header received
0806 03:51:50.256579 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:50.256580 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:50.256590 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:50.256646 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:50.257555 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.258139 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.259408 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.259634 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.259690 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:50.262160 (+ 2470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:50.262162 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:50.262162 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:50.262171 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:50.263731 (+ 1560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:50.264144 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:50.264145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:50.264146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:50.264215 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:50.264422 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:50.264423 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:50.264424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:50.264568 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:50.264640 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:50.264859 (+ 219us) server_negotiation.cc:299] Negotiation successful
0806 03:51:50.264950 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:51:50.265122 16485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:50.255814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40917 (local address 127.12.15.65:56067)
0806 03:51:50.255982 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:50.256023 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:50.256502 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:50.256703 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:50.256706 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:50.257273 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:50.257460 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:50.257465 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.258233 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.258235 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:50.259324 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:50.259327 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.259715 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.259716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:50.259848 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:50.260361 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:50.260371 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:50.262030 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:50.263829 (+ 1799us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:50.263837 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:50.263839 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:50.264071 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:50.264276 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:50.264276 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:50.264277 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:50.264369 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:50.264626 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:50.264629 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:50.264742 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 03:51:50.264820 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 03:51:50.264937 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":90,"threads_started":1}
I20260806 03:51:50.265619 16483 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40917
I20260806 03:51:50.265713 16483 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:50.265887 16483 heartbeater.cc:511] Master 127.12.15.126:40917 requested a full tablet report, sending...
I20260806 03:51:50.266482 16164 ts_manager.cc:194] Registered new tserver with Master: 97345799da8a4b71a5bcecff8960aaa6 (127.12.15.65:37981)
I20260806 03:51:50.267378 16164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:56067
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:51:50.282704 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40917
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:50.362423 16489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:50.362633 16489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:50.362654 16489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:50.363766 16489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:50.363804 16489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:50.363837 16489 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:50.365238 16489 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:50.365460 16489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:50.365676 16489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:50.370353 16496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:50.370381 16499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:50.370431 16489 server_base.cc:1034] running on GCE node
W20260806 03:51:50.370396 16501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:50.370692 16489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:50.370913 16489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:50.372068 16489 hybrid_clock.cc:648] HybridClock initialized: now 1785988310372060 us; error 28 us; skew 500 ppm
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988310, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:50.374418 16489 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:51:50.374825 16489 webserver.cc:460] Webserver started at http://127.12.15.66:38585/ using document root <none> and password file <none>
I20260806 03:51:50.374993 16489 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:50.375034 16489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:50.375123 16489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:50.376060 16489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "1792240b49e54983944c08a393e17737"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "ddca850387ad1312e5073fbff47a478d"
server_key_iv: "f04ceff49e4adb83839c5eab984d2936"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.376343 16489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "1792240b49e54983944c08a393e17737"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "ddca850387ad1312e5073fbff47a478d"
server_key_iv: "f04ceff49e4adb83839c5eab984d2936"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.377573 16489 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:50.378193 16519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.378341 16489 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:50.378419 16489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "1792240b49e54983944c08a393e17737"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "ddca850387ad1312e5073fbff47a478d"
server_key_iv: "f04ceff49e4adb83839c5eab984d2936"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.378484 16489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:50.391095 16489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:50.393112 16489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:50.393239 16489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:50.393450 16489 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:50.393754 16489 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:50.393787 16489 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.393814 16489 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:50.393839 16489 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.399297 16489 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:40847
I20260806 03:51:50.399338 16717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:40847 every 8 connection(s)
I20260806 03:51:50.399648 16489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988310, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:50.406919 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 16489
I20260806 03:51:50.407023 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:50.407229 12349 external_mini_cluster.cc:1428] Setting key f7e0af29ad873938cf2d1595de506da7
I20260806 03:51:50.408260 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:50.400312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:51017 (local address 127.12.15.126:40917)
0806 03:51:50.400367 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:50.400369 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:50.400543 (+ 174us) server_negotiation.cc:407] Connection header received
0806 03:51:50.401037 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:50.401039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:50.401050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:50.401099 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:50.401997 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.402532 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.403481 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.403680 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.403737 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:50.405729 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:50.405730 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:50.405731 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:50.405737 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:50.406865 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:50.407313 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:50.407314 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:50.407314 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:50.407379 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:50.407632 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:50.407633 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:50.407633 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:50.407772 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:50.407831 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:50.408091 (+ 260us) server_negotiation.cc:299] Negotiation successful
0806 03:51:50.408172 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:51:50.408432 16723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:50.400273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40917 (local address 127.12.15.66:51017)
0806 03:51:50.400437 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:50.400471 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:50.400948 (+ 477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:50.401177 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:50.401181 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:50.401727 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:50.401910 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:50.401914 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.402617 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.402618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:50.403403 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:50.403405 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.403749 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.403750 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:50.403865 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:50.404301 (+ 436us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:50.404308 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:50.405649 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:50.406936 (+ 1287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:50.406943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:50.406946 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:50.407240 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:50.407432 (+ 192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:50.407433 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:50.407433 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:50.407578 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:50.407845 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:50.407851 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:50.407976 (+ 125us) client_negotiation.cc:769] Sending connection context
0806 03:51:50.408062 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 03:51:50.408221 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":89,"threads_started":1}
I20260806 03:51:50.409008 16719 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40917
I20260806 03:51:50.409094 16719 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:50.409266 16719 heartbeater.cc:511] Master 127.12.15.126:40917 requested a full tablet report, sending...
I20260806 03:51:50.409672 16164 ts_manager.cc:194] Registered new tserver with Master: 1792240b49e54983944c08a393e17737 (127.12.15.66:40847)
I20260806 03:51:50.410002 16164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:51017
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:51:50.430440 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40917
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44471
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:50.509855 16727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:50.510052 16727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:50.510083 16727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:50.511204 16727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:50.511245 16727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:50.511278 16727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:50.512655 16727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44471
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:50.512876 16727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:50.513069 16727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:50.518210 16737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:50.518232 16739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:50.518328 16727 server_base.cc:1034] running on GCE node
W20260806 03:51:50.518250 16742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:50.518627 16727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:50.518860 16727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:50.519994 16727 hybrid_clock.cc:648] HybridClock initialized: now 1785988310519983 us; error 31 us; skew 500 ppm
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16089](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988310, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:50.522310 16727 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:51:50.522702 16727 webserver.cc:460] Webserver started at http://127.12.15.67:44809/ using document root <none> and password file <none>
I20260806 03:51:50.522895 16727 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:50.522941 16727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:50.523064 16727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:50.524021 16727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "639b62c8da954c069c3896e6e723db05"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "dbdb3979df296c70217f5fd43bb6c7f8"
server_key_iv: "bfda1a3e8928f571a829e00c21b26a90"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.524346 16727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "639b62c8da954c069c3896e6e723db05"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "dbdb3979df296c70217f5fd43bb6c7f8"
server_key_iv: "bfda1a3e8928f571a829e00c21b26a90"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.525655 16727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:50.526311 16755 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.526453 16727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.526531 16727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "639b62c8da954c069c3896e6e723db05"
format_stamp: "Formatted at 2026-08-06 03:51:50 on dist-test-slave-bqcl"
server_key: "dbdb3979df296c70217f5fd43bb6c7f8"
server_key_iv: "bfda1a3e8928f571a829e00c21b26a90"
server_key_version: "encryptionkey@0"
I20260806 03:51:50.526597 16727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:50.537818 16727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:50.539579 16727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:50.539708 16727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:50.539891 16727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:50.540156 16727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:50.540189 16727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.540215 16727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:50.540241 16727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:50.545759 16727 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:42327
I20260806 03:51:50.545796 16934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:42327 every 8 connection(s)
I20260806 03:51:50.546073 16727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16089](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988310, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:50.554189 16225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:50.546809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:34653 (local address 127.12.15.126:40917)
0806 03:51:50.546856 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:50.546858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:50.547041 (+ 183us) server_negotiation.cc:407] Connection header received
0806 03:51:50.547456 (+ 415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:50.547457 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:50.547467 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:50.547516 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:50.548361 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.548911 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.549831 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.550032 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.550087 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:50.551992 (+ 1905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:50.551993 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:50.551993 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:50.552000 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:50.553203 (+ 1203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:50.553489 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:50.553489 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:50.553490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:50.553539 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:50.553703 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:50.553703 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:50.553703 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:50.553794 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:50.553843 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:50.554035 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 03:51:50.554089 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:50.554335 16939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:50.546793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40917 (local address 127.12.15.67:34653)
0806 03:51:50.546932 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:50.546964 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:50.547373 (+ 409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:50.547568 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:50.547571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:50.548113 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:50.548292 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:50.548297 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.548997 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.548998 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:50.549760 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:50.549762 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:50.550093 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:50.550094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:50.550275 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:50.550651 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:50.550659 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:50.551920 (+ 1261us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:50.553260 (+ 1340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:50.553263 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:50.553264 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:50.553440 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:50.553580 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:50.553581 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:50.553581 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:50.553661 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:50.553838 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:50.553840 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:50.553939 (+ 99us) client_negotiation.cc:769] Sending connection context
0806 03:51:50.554010 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 03:51:50.554119 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":80,"threads_started":1}
I20260806 03:51:50.554775 16937 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40917
I20260806 03:51:50.554857 16937 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:50.554987 16937 heartbeater.cc:511] Master 127.12.15.126:40917 requested a full tablet report, sending...
I20260806 03:51:50.555285 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 16727
I20260806 03:51:50.555328 16164 ts_manager.cc:194] Registered new tserver with Master: 639b62c8da954c069c3896e6e723db05 (127.12.15.67:42327)
I20260806 03:51:50.555373 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:50.555538 12349 external_mini_cluster.cc:1428] Setting key f1f11353f503465a0b5575fe119cedd2
I20260806 03:51:50.555716 16164 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:34653
I20260806 03:51:50.556421 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 03:51:50.569114 16225 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:50.568312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49632 (local address 127.12.15.126:40917)
0806 03:51:50.568364 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:50.568365 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:50.568451 (+ 86us) server_negotiation.cc:407] Connection header received
0806 03:51:50.568510 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:50.568511 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:50.568520 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:50.568561 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:50.569049 (+ 488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49632: BlockingRecv error: recv got EOF from 127.0.0.1:49632 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:51:50.569929 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 16264
I20260806 03:51:50.593344 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 16489
I20260806 03:51:50.604517 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 16727
I20260806 03:51:50.615201 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 16112
2026-08-06T03:51:50Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1921 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16955](info): set up 2 sockets
Aug 06 03:51:50 dist-test-slave-bqcl krb5kdc[16955](info): commencing operation
krb5kdc: starting...
W20260806 03:51:51.749814 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:51 dist-test-slave-bqcl krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:51.860034 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33061
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38027
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:33061
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:51.938563 16978 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:51.938755 16978 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:51.938784 16978 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:51.939877 16978 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:51.939915 16978 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:51.939934 16978 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:51.939949 16978 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:51.939963 16978 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:51.941390 16978 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38027
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:33061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16978
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:51.941630 16978 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:51.941835 16978 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:51.946429 16989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:51.946430 16987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:51.946521 16978 server_base.cc:1034] running on GCE node
W20260806 03:51:51.946426 16986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:51.946882 16978 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:51.947144 16978 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:51.948278 16978 hybrid_clock.cc:648] HybridClock initialized: now 1785988311948265 us; error 29 us; skew 500 ppm
Aug 06 03:51:51 dist-test-slave-bqcl krb5kdc[16955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988311, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:51.953699 16978 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:51.954210 16978 webserver.cc:460] Webserver started at http://127.12.15.126:39979/ using document root <none> and password file <none>
I20260806 03:51:51.954425 16978 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:51.954473 16978 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:51.954597 16978 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:51.955641 16978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "679768761b6b4926976581fd2a2b554d"
format_stamp: "Formatted at 2026-08-06 03:51:51 on dist-test-slave-bqcl"
server_key: "0c1e614c5f4293e2ebaddf5b8b0d452e"
server_key_iv: "3661a5503f65b520e20e7a8f69650625"
server_key_version: "encryptionkey@0"
I20260806 03:51:51.955958 16978 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "679768761b6b4926976581fd2a2b554d"
format_stamp: "Formatted at 2026-08-06 03:51:51 on dist-test-slave-bqcl"
server_key: "0c1e614c5f4293e2ebaddf5b8b0d452e"
server_key_iv: "3661a5503f65b520e20e7a8f69650625"
server_key_version: "encryptionkey@0"
I20260806 03:51:51.957132 16978 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:51.957743 17004 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:51.957891 16978 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:51.957973 16978 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "679768761b6b4926976581fd2a2b554d"
format_stamp: "Formatted at 2026-08-06 03:51:51 on dist-test-slave-bqcl"
server_key: "0c1e614c5f4293e2ebaddf5b8b0d452e"
server_key_iv: "3661a5503f65b520e20e7a8f69650625"
server_key_version: "encryptionkey@0"
I20260806 03:51:51.958033 16978 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:51.984304 16978 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:51.986418 16978 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:51.986550 16978 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:51.990470 16978 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:33061
I20260806 03:51:51.990510 17089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:33061 every 8 connection(s)
I20260806 03:51:51.990993 16978 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:51.991597 17090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:51.993728 17090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d: Bootstrap starting.
I20260806 03:51:51.994035 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 16978
I20260806 03:51:51.994115 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:51.994298 12349 external_mini_cluster.cc:1428] Setting key 26344b667568b9c8c187f571a1276f04
I20260806 03:51:51.994412 17090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:51.994755 17090 log.cc:824] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:51.995426 17090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d: No bootstrap required, opened a new log
I20260806 03:51:51.997531 17090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679768761b6b4926976581fd2a2b554d" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33061 } }
I20260806 03:51:51.997696 17090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:51.997730 17090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 679768761b6b4926976581fd2a2b554d, State: Initialized, Role: FOLLOWER
I20260806 03:51:51.997844 17090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679768761b6b4926976581fd2a2b554d" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33061 } }
I20260806 03:51:51.997911 17090 raft_consensus.cc:388] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:51.997943 17090 raft_consensus.cc:482] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:51.997984 17090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:51.998598 17090 raft_consensus.cc:504] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679768761b6b4926976581fd2a2b554d" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33061 } }
I20260806 03:51:51.998700 17090 leader_election.cc:302] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 679768761b6b4926976581fd2a2b554d; no voters:
I20260806 03:51:51.998865 17090 leader_election.cc:288] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:51.998884 17098 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:51.999140 17090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:51.999315 17098 raft_consensus.cc:686] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [term 1 LEADER]: Becoming Leader. State: Replica: 679768761b6b4926976581fd2a2b554d, State: Running, Role: LEADER
I20260806 03:51:51.999454 17098 consensus_queue.cc:237] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679768761b6b4926976581fd2a2b554d" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33061 } }
I20260806 03:51:51.999861 17099 sys_catalog.cc:455] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "679768761b6b4926976581fd2a2b554d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679768761b6b4926976581fd2a2b554d" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33061 } } }
I20260806 03:51:51.999951 17099 sys_catalog.cc:458] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:52.000106 17101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 679768761b6b4926976581fd2a2b554d. Latest consensus state: current_term: 1 leader_uuid: "679768761b6b4926976581fd2a2b554d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "679768761b6b4926976581fd2a2b554d" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33061 } } }
I20260806 03:51:52.000155 17101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:52.000157 17115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:52.000711 17115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 03:51:51 dist-test-slave-bqcl krb5kdc[16955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988311, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:52.002830 17115 catalog_manager.cc:1347] Generated new cluster ID: e0eeb57eef384f0cb7ee13a61ad37b03
I20260806 03:51:52.002880 17115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:52.004752 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:51.994748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42964 (local address 127.12.15.126:33061)
0806 03:51:51.994951 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:51.994961 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:51.994997 (+ 36us) server_negotiation.cc:407] Connection header received
0806 03:51:51.995440 (+ 443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:51.995447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:51.995508 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:51.995632 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:51.996322 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:51.997157 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:51.998200 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:51.998606 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:51.998709 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:52.001399 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:52.001403 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:52.001403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:52.001415 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:52.003216 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:52.003575 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:52.003579 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:52.003579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:52.003691 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:52.003866 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:52.003867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:52.003867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:52.004164 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:52.004343 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:52.004413 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 03:51:52.004544 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":61,"threads_started":1}
I20260806 03:51:52.005628 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:52.011047 17115 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:52.011723 17115 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:52.020058 17115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 679768761b6b4926976581fd2a2b554d: Generated new TSK 0
I20260806 03:51:52.020229 17115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:52.022004 17095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:52.018218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.12.15.126:33061)
0806 03:51:52.018256 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:52.018258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:52.018428 (+ 170us) server_negotiation.cc:407] Connection header received
0806 03:51:52.018486 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:52.018487 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:52.018497 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:52.018565 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:52.018909 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:52.019694 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:52.020601 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:52.020929 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:52.020993 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:52.021566 (+ 573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:52.021568 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:52.021568 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:52.021771 (+ 203us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:52.021790 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:52.021798 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:52.021858 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:52.021902 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 03:51:52.021927 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:51:52.024394 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 16978
2026-08-06T03:51:52Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1415 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:52 dist-test-slave-bqcl krb5kdc[17150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:52 dist-test-slave-bqcl krb5kdc[17150](info): set up 2 sockets
Aug 06 03:51:52 dist-test-slave-bqcl krb5kdc[17150](info): commencing operation
krb5kdc: starting...
W20260806 03:51:53.158423 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:53.274281 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33253
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:33253
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:53.354166 17167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:53.354372 17167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:53.354394 17167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:53.355494 17167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:53.355535 17167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:53.355547 17167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:53.355559 17167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:53.355571 17167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:53.356987 17167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:33253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:53.357187 17167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:53.357365 17167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:51:53.362215 17167 server_base.cc:1034] running on GCE node
W20260806 03:51:53.362218 17184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:53.362243 17185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:53.362362 17187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:53.362812 17167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:53.363098 17167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:53.364260 17167 hybrid_clock.cc:648] HybridClock initialized: now 1785988313364245 us; error 29 us; skew 500 ppm
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:53.366739 17167 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:53.367170 17167 webserver.cc:460] Webserver started at http://127.12.15.126:33461/ using document root <none> and password file <none>
I20260806 03:51:53.367388 17167 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:53.367436 17167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:53.367566 17167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:53.368643 17167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "f1021b3641c7443b8caa870a87c687cf"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "70d1a49e2585cacf877aeba12b49c686"
server_key_iv: "e5f54b60be157fa3886254f850d2ac73"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.368975 17167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "f1021b3641c7443b8caa870a87c687cf"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "70d1a49e2585cacf877aeba12b49c686"
server_key_iv: "e5f54b60be157fa3886254f850d2ac73"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.370203 17167 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:53.370882 17200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.371025 17167 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:53.371102 17167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "f1021b3641c7443b8caa870a87c687cf"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "70d1a49e2585cacf877aeba12b49c686"
server_key_iv: "e5f54b60be157fa3886254f850d2ac73"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.371177 17167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:53.393962 17167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:53.395807 17167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:53.395946 17167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:53.399807 17167 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:33253
I20260806 03:51:53.399880 17277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:33253 every 8 connection(s)
I20260806 03:51:53.400192 17167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:53.401338 17278 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:53.403602 17278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf: Bootstrap starting.
I20260806 03:51:53.404138 17278 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:53.404368 17278 log.cc:824] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:53.404994 17278 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf: No bootstrap required, opened a new log
I20260806 03:51:53.406553 17278 raft_consensus.cc:348] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1021b3641c7443b8caa870a87c687cf" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33253 } }
I20260806 03:51:53.406689 17278 raft_consensus.cc:374] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:53.406718 17278 raft_consensus.cc:729] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1021b3641c7443b8caa870a87c687cf, State: Initialized, Role: FOLLOWER
I20260806 03:51:53.406826 17278 consensus_queue.cc:260] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1021b3641c7443b8caa870a87c687cf" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33253 } }
I20260806 03:51:53.406875 17278 raft_consensus.cc:388] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:53.406894 17278 raft_consensus.cc:482] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:53.406922 17278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:53.407739 17278 raft_consensus.cc:504] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1021b3641c7443b8caa870a87c687cf" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33253 } }
I20260806 03:51:53.407825 17278 leader_election.cc:302] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f1021b3641c7443b8caa870a87c687cf; no voters:
I20260806 03:51:53.407943 17278 leader_election.cc:288] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:53.407984 17281 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:53.408118 17281 raft_consensus.cc:686] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [term 1 LEADER]: Becoming Leader. State: Replica: f1021b3641c7443b8caa870a87c687cf, State: Running, Role: LEADER
I20260806 03:51:53.408187 17278 sys_catalog.cc:564] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:53.408237 17281 consensus_queue.cc:237] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1021b3641c7443b8caa870a87c687cf" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33253 } }
I20260806 03:51:53.408581 17284 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f1021b3641c7443b8caa870a87c687cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1021b3641c7443b8caa870a87c687cf" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33253 } } }
I20260806 03:51:53.408593 17285 sys_catalog.cc:455] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader f1021b3641c7443b8caa870a87c687cf. Latest consensus state: current_term: 1 leader_uuid: "f1021b3641c7443b8caa870a87c687cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f1021b3641c7443b8caa870a87c687cf" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33253 } } }
I20260806 03:51:53.408705 17284 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:53.408776 17285 sys_catalog.cc:458] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:53.409164 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 17167
I20260806 03:51:53.409263 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:53.409430 12349 external_mini_cluster.cc:1428] Setting key 5afb8eb40fafe0e5ad50c18b0163ecac
I20260806 03:51:53.409601 17301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:53.410333 17301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:53.413116 17301 catalog_manager.cc:1347] Generated new cluster ID: 2f7b84978f3d4dcc833f8f71fbacec19
I20260806 03:51:53.413241 17301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:53.419250 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:53.409880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36690 (local address 127.12.15.126:33253)
0806 03:51:53.410025 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:53.410034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:53.410064 (+ 30us) server_negotiation.cc:407] Connection header received
0806 03:51:53.410250 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:53.410257 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:53.410314 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:53.410452 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:53.411276 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.412634 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.413406 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.413840 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.413934 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:53.416077 (+ 2143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:53.416080 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:53.416080 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:53.416090 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:53.417799 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.418156 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.418159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.418159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.418255 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.418436 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.418437 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.418437 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.418692 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:53.418848 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:53.418945 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 03:51:53.419083 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":68,"threads_started":1}
I20260806 03:51:53.456203 17301 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:53.456781 17301 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:53.468482 17301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f1021b3641c7443b8caa870a87c687cf: Generated new TSK 0
I20260806 03:51:53.468715 17301 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:51:53.501827 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33253
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:53.583630 17326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:53.583832 17326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:53.583853 17326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:53.584962 17326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:53.585006 17326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:53.585035 17326 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:53.586484 17326 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17326
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:53.586709 17326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:53.586916 17326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:53.591730 17338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:53.591698 17333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:53.591709 17335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:53.591782 17326 server_base.cc:1034] running on GCE node
I20260806 03:51:53.592096 17326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:53.592348 17326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:53.593484 17326 hybrid_clock.cc:648] HybridClock initialized: now 1785988313593475 us; error 31 us; skew 500 ppm
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:53.595830 17326 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:51:53.596297 17326 webserver.cc:460] Webserver started at http://127.12.15.65:45825/ using document root <none> and password file <none>
I20260806 03:51:53.596486 17326 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:53.596532 17326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:53.596652 17326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:53.597613 17326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "e8917dd478fa453593c7633a2ca003ce"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "d9611c380529b9cad955e9c3b0c82108"
server_key_iv: "a3cc93319740ae174db331b783adad66"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.597934 17326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "e8917dd478fa453593c7633a2ca003ce"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "d9611c380529b9cad955e9c3b0c82108"
server_key_iv: "a3cc93319740ae174db331b783adad66"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.599205 17326 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:53.599852 17348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.600009 17326 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:53.600090 17326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "e8917dd478fa453593c7633a2ca003ce"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "d9611c380529b9cad955e9c3b0c82108"
server_key_iv: "a3cc93319740ae174db331b783adad66"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.600152 17326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:53.618964 17326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:53.620893 17326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:53.621029 17326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:53.621274 17326 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:53.621585 17326 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:53.621623 17326 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.621666 17326 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:53.621695 17326 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.627106 17326 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:44395
I20260806 03:51:53.627167 17526 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:44395 every 8 connection(s)
I20260806 03:51:53.627470 17326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:53.636288 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:53.628116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:49377 (local address 127.12.15.126:33253)
0806 03:51:53.628161 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:53.628162 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:53.628370 (+ 208us) server_negotiation.cc:407] Connection header received
0806 03:51:53.628877 (+ 507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:53.628878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:53.628891 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:53.628938 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:53.629859 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.630508 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.631482 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.631674 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.631730 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:53.633754 (+ 2024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:53.633756 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:53.633756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:53.633763 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:53.635040 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.635408 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.635409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.635409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.635469 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.635693 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.635694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.635694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.635801 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:53.635855 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:53.636111 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 03:51:53.636197 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:51:53.636451 17534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:53.628098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33253 (local address 127.12.15.65:49377)
0806 03:51:53.628255 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:53.628291 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:53.628782 (+ 491us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:53.629022 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:53.629026 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:53.629569 (+ 543us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:53.629764 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:53.629769 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.630596 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.630597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:53.631401 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:53.631403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.631743 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.631744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:53.631866 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:53.632304 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:53.632313 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:53.633674 (+ 1361us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:53.635123 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:53.635131 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:53.635133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:53.635341 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:53.635536 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:53.635537 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:53.635537 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:53.635635 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:53.635856 (+ 221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:53.635864 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:53.635991 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 03:51:53.636081 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 03:51:53.636233 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":72,"threads_started":1}
I20260806 03:51:53.636924 17530 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33253
I20260806 03:51:53.637010 17530 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:53.637190 17530 heartbeater.cc:511] Master 127.12.15.126:33253 requested a full tablet report, sending...
I20260806 03:51:53.637529 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 17326
I20260806 03:51:53.637629 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:53.637769 17231 ts_manager.cc:194] Registered new tserver with Master: e8917dd478fa453593c7633a2ca003ce (127.12.15.65:44395)
I20260806 03:51:53.637830 12349 external_mini_cluster.cc:1428] Setting key f34b36122f0393e0f37fc3e99ae20b22
I20260806 03:51:53.638484 17231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:49377
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:51:53.660805 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33253
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:53.739745 17537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:53.739926 17537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:53.739955 17537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:53.741025 17537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:53.741063 17537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:53.741096 17537 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:53.742473 17537 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:53.742689 17537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:53.742887 17537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:53.747478 17544 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:53.747648 17537 server_base.cc:1034] running on GCE node
W20260806 03:51:53.747701 17545 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:53.747785 17548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:53.748000 17537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:53.748230 17537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:53.749368 17537 hybrid_clock.cc:648] HybridClock initialized: now 1785988313749352 us; error 33 us; skew 500 ppm
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:53.751778 17537 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:51:53.752192 17537 webserver.cc:460] Webserver started at http://127.12.15.66:35965/ using document root <none> and password file <none>
I20260806 03:51:53.752378 17537 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:53.752417 17537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:53.752504 17537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:53.753458 17537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "b15f4ecd114d4dcc93e831c48ddc50d2"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "4b5718c80f0886e2bde70a7b8f849b21"
server_key_iv: "39e6728211a972611d9958e025c1f73e"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.753746 17537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "b15f4ecd114d4dcc93e831c48ddc50d2"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "4b5718c80f0886e2bde70a7b8f849b21"
server_key_iv: "39e6728211a972611d9958e025c1f73e"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.755048 17537 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:53.755666 17560 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.755815 17537 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.755877 17537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "b15f4ecd114d4dcc93e831c48ddc50d2"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "4b5718c80f0886e2bde70a7b8f849b21"
server_key_iv: "39e6728211a972611d9958e025c1f73e"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.755940 17537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:53.774302 17537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:53.776042 17537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:53.776149 17537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:53.776333 17537 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:53.776610 17537 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:53.776644 17537 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.776665 17537 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:53.776687 17537 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.782088 17537 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:32857
I20260806 03:51:53.782179 17748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:32857 every 8 connection(s)
I20260806 03:51:53.782603 17537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:51:53.785383 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 17537
I20260806 03:51:53.785486 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:53.785633 12349 external_mini_cluster.cc:1428] Setting key 617d32e22522acc897cd2051a5aeb10b
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:53.792122 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:53.783404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:51957 (local address 127.12.15.126:33253)
0806 03:51:53.783460 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:53.783462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:53.783673 (+ 211us) server_negotiation.cc:407] Connection header received
0806 03:51:53.784135 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:53.784135 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:53.784145 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:53.784197 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:53.785040 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.785720 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.786887 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.787097 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.787153 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:53.789431 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:53.789433 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:53.789433 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:53.789440 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:53.790720 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.791162 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.791163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.791163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.791208 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.791486 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.791488 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.791488 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.791614 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:53.791678 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:53.791984 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 03:51:53.792058 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:51:53.792359 17756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:53.783395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33253 (local address 127.12.15.66:51957)
0806 03:51:53.783559 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:53.783594 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:53.784057 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:53.784249 (+ 192us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:53.784252 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:53.784792 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:53.784970 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:53.784974 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.785889 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.785889 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:53.786805 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:53.786808 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.787168 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.787168 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:53.787297 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:53.787749 (+ 452us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:53.787760 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:53.789330 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:53.790792 (+ 1462us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:53.790800 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:53.790803 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:53.791092 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:53.791264 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:53.791264 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:53.791265 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:53.791400 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:53.791690 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:53.791697 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:53.791858 (+ 161us) client_negotiation.cc:769] Sending connection context
0806 03:51:53.791977 (+ 119us) client_negotiation.cc:240] Negotiation successful
0806 03:51:53.792140 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":83,"threads_started":1}
I20260806 03:51:53.792873 17749 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33253
I20260806 03:51:53.792959 17749 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:53.793123 17749 heartbeater.cc:511] Master 127.12.15.126:33253 requested a full tablet report, sending...
I20260806 03:51:53.793494 17231 ts_manager.cc:194] Registered new tserver with Master: b15f4ecd114d4dcc93e831c48ddc50d2 (127.12.15.66:32857)
I20260806 03:51:53.793883 17231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:51957
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:51:53.810055 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33253
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38327
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:53.889385 17760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:53.889565 17760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:53.889585 17760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:53.890686 17760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:53.890723 17760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:53.890748 17760 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:53.892092 17760 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38327
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:53.892262 17760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:53.892428 17760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:53.897063 17770 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:53.897213 17760 server_base.cc:1034] running on GCE node
W20260806 03:51:53.897053 17772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:53.897102 17769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:53.897460 17760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:53.897645 17760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:53.898780 17760 hybrid_clock.cc:648] HybridClock initialized: now 1785988313898767 us; error 30 us; skew 500 ppm
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:53.901119 17760 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:51:53.901526 17760 webserver.cc:460] Webserver started at http://127.12.15.67:35907/ using document root <none> and password file <none>
I20260806 03:51:53.901732 17760 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:53.901778 17760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:53.901901 17760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:53.903021 17760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "a507861cac3d49afab0a935a516bdac6"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "9c5d129271a6e4e13e4d316b0b6bd04c"
server_key_iv: "1ce0cde0b6b8571c372f4d1e28334c6f"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.903366 17760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "a507861cac3d49afab0a935a516bdac6"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "9c5d129271a6e4e13e4d316b0b6bd04c"
server_key_iv: "1ce0cde0b6b8571c372f4d1e28334c6f"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.904727 17760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:53.905540 17786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.905747 17760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:53.905828 17760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "a507861cac3d49afab0a935a516bdac6"
format_stamp: "Formatted at 2026-08-06 03:51:53 on dist-test-slave-bqcl"
server_key: "9c5d129271a6e4e13e4d316b0b6bd04c"
server_key_iv: "1ce0cde0b6b8571c372f4d1e28334c6f"
server_key_version: "encryptionkey@0"
I20260806 03:51:53.905893 17760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:53.915993 17760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:53.917646 17760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:53.917768 17760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:53.917955 17760 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:53.918291 17760 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:53.918326 17760 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.918354 17760 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:53.918380 17760 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:53.923938 17760 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:39139
I20260806 03:51:53.924008 17959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:39139 every 8 connection(s)
I20260806 03:51:53.924279 17760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:53.932562 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:53.924934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:42219 (local address 127.12.15.126:33253)
0806 03:51:53.925005 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:53.925008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:53.925154 (+ 146us) server_negotiation.cc:407] Connection header received
0806 03:51:53.925690 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:53.925691 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:53.925700 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:53.925745 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:53.926678 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.927152 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.928155 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.928333 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.928377 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:53.930410 (+ 2033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:53.930411 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:53.930411 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:53.930419 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:53.931523 (+ 1104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.931850 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.931850 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.931851 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.931902 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.932077 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.932077 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.932078 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.932176 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:53.932227 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:53.932415 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 03:51:53.932498 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:51:53.932662 17967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:53.924894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33253 (local address 127.12.15.67:42219)
0806 03:51:53.925042 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:53.925077 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:53.925603 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:53.925805 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:53.925809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:53.926406 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:53.926607 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:53.926613 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.927248 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.927249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:53.928088 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:53.928091 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.928388 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.928389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:53.928507 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:53.928906 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:53.928914 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:53.930324 (+ 1410us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:53.931588 (+ 1264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:53.931594 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:53.931595 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:53.931794 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:53.931949 (+ 155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:53.931950 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:53.931950 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:53.932034 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:53.932221 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:53.932223 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:53.932324 (+ 101us) client_negotiation.cc:769] Sending connection context
0806 03:51:53.932392 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 03:51:53.932501 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260806 03:51:53.933108 17961 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33253
I20260806 03:51:53.933199 17961 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:53.933377 17961 heartbeater.cc:511] Master 127.12.15.126:33253 requested a full tablet report, sending...
I20260806 03:51:53.933758 17231 ts_manager.cc:194] Registered new tserver with Master: a507861cac3d49afab0a935a516bdac6 (127.12.15.67:39139)
I20260806 03:51:53.934116 17231 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:42219
I20260806 03:51:53.934368 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 17760
I20260806 03:51:53.934446 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:53.934592 12349 external_mini_cluster.cc:1428] Setting key b67738b85b8ccecb14671b412141fa66
I20260806 03:51:53.935386 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 06 03:51:53 dist-test-slave-bqcl krb5kdc[17150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988313, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:53.957897 17317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:53.951462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36700 (local address 127.12.15.126:33253)
0806 03:51:53.951525 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:53.951528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:53.951604 (+ 76us) server_negotiation.cc:407] Connection header received
0806 03:51:53.951666 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:53.951666 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:53.951677 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:53.951731 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:53.952253 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.952775 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.953475 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:53.953669 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:53.953725 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:53.955799 (+ 2074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:53.955800 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:53.955800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:53.955808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:53.956878 (+ 1070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.957168 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.957169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.957169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.957229 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:53.957388 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:53.957389 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:53.957389 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:53.957528 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:53.957614 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:53.957731 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 03:51:53.957807 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260806 03:51:53.958150 17231 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:36700
I20260806 03:51:53.958621 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 17326
I20260806 03:51:53.971745 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 17537
I20260806 03:51:53.986274 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 17760
I20260806 03:51:54.004976 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 17167
2026-08-06T03:51:54Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1976 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:54 dist-test-slave-bqcl krb5kdc[17986](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:54 dist-test-slave-bqcl krb5kdc[17986](info): set up 2 sockets
Aug 06 03:51:54 dist-test-slave-bqcl krb5kdc[17986](info): commencing operation
krb5kdc: starting...
W20260806 03:51:55.128299 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:55.240976 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:42653
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:42653
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:55.321475 18004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:55.321672 18004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:55.321693 18004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:55.322784 18004 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:55.322819 18004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:55.322832 18004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:55.322844 18004 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:55.322856 18004 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:55.324302 18004 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:42653
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:42653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18004
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:55.324481 18004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:55.324653 18004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:55.329069 18010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:55.329095 18018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:55.329125 18013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:55.329402 18004 server_base.cc:1034] running on GCE node
I20260806 03:51:55.329660 18004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:55.329937 18004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:55.331122 18004 hybrid_clock.cc:648] HybridClock initialized: now 1785988315331105 us; error 26 us; skew 500 ppm
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:55.333666 18004 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:55.334102 18004 webserver.cc:460] Webserver started at http://127.12.15.126:38977/ using document root <none> and password file <none>
I20260806 03:51:55.334342 18004 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:55.334391 18004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:55.334512 18004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:55.335503 18004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "93bb5501d9844ed59f41cd1933cc9d74"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "c16b02e28541f9752e7006d5a6d54375"
server_key_iv: "2db2ca435b0306d1ca1de6da54bcc25c"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.335825 18004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "93bb5501d9844ed59f41cd1933cc9d74"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "c16b02e28541f9752e7006d5a6d54375"
server_key_iv: "2db2ca435b0306d1ca1de6da54bcc25c"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.336997 18004 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:55.337699 18027 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.337841 18004 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:55.337924 18004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "93bb5501d9844ed59f41cd1933cc9d74"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "c16b02e28541f9752e7006d5a6d54375"
server_key_iv: "2db2ca435b0306d1ca1de6da54bcc25c"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.337992 18004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:55.354975 18004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:55.356961 18004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:55.357100 18004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:55.360867 18004 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:42653
I20260806 03:51:55.360904 18106 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:42653 every 8 connection(s)
I20260806 03:51:55.361359 18004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:55.362046 18108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:55.364231 18108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74: Bootstrap starting.
I20260806 03:51:55.364792 18108 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:55.365051 18108 log.cc:824] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:55.365320 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 18004
I20260806 03:51:55.365423 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:55.365617 12349 external_mini_cluster.cc:1428] Setting key eb4128c8af6bd35f045a2cff8cff695f
I20260806 03:51:55.365825 18108 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74: No bootstrap required, opened a new log
I20260806 03:51:55.368129 18108 raft_consensus.cc:348] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93bb5501d9844ed59f41cd1933cc9d74" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42653 } }
I20260806 03:51:55.368261 18108 raft_consensus.cc:374] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:55.368291 18108 raft_consensus.cc:729] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 93bb5501d9844ed59f41cd1933cc9d74, State: Initialized, Role: FOLLOWER
I20260806 03:51:55.368415 18108 consensus_queue.cc:260] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93bb5501d9844ed59f41cd1933cc9d74" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42653 } }
I20260806 03:51:55.368489 18108 raft_consensus.cc:388] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:55.368535 18108 raft_consensus.cc:482] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:55.368582 18108 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:55.369082 18108 raft_consensus.cc:504] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93bb5501d9844ed59f41cd1933cc9d74" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42653 } }
I20260806 03:51:55.369176 18108 leader_election.cc:302] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 93bb5501d9844ed59f41cd1933cc9d74; no voters:
I20260806 03:51:55.369323 18108 leader_election.cc:288] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:55.369369 18114 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:55.369540 18114 raft_consensus.cc:686] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [term 1 LEADER]: Becoming Leader. State: Replica: 93bb5501d9844ed59f41cd1933cc9d74, State: Running, Role: LEADER
I20260806 03:51:55.369556 18108 sys_catalog.cc:564] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:55.369722 18114 consensus_queue.cc:237] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93bb5501d9844ed59f41cd1933cc9d74" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42653 } }
I20260806 03:51:55.370055 18116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "93bb5501d9844ed59f41cd1933cc9d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93bb5501d9844ed59f41cd1933cc9d74" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42653 } } }
I20260806 03:51:55.370139 18116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:55.370344 18127 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:55.370850 18127 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:55.371169 18115 sys_catalog.cc:455] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 93bb5501d9844ed59f41cd1933cc9d74. Latest consensus state: current_term: 1 leader_uuid: "93bb5501d9844ed59f41cd1933cc9d74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "93bb5501d9844ed59f41cd1933cc9d74" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42653 } } }
I20260806 03:51:55.371250 18115 sys_catalog.cc:458] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:55.372354 18127 catalog_manager.cc:1347] Generated new cluster ID: 1417a43b79a64f51891ac2c8dd9185d1
I20260806 03:51:55.372404 18127 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:55.377164 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.366099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37432 (local address 127.12.15.126:42653)
0806 03:51:55.366295 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:55.366304 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:55.366327 (+ 23us) server_negotiation.cc:407] Connection header received
0806 03:51:55.367012 (+ 685us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:55.367019 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:55.367066 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:55.367187 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:55.367786 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.368759 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.369636 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.370319 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.370402 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.373868 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:55.373872 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:55.373872 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:55.373884 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:55.375695 (+ 1811us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.376065 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.376067 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.376068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.376160 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.376359 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.376359 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.376360 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.376625 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:55.376764 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:55.376865 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 03:51:55.376983 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":105,"threads_started":1}
I20260806 03:51:55.384815 18127 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:55.385294 18127 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:55.388211 18127 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 93bb5501d9844ed59f41cd1933cc9d74: Generated new TSK 0
I20260806 03:51:55.388371 18127 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:51:55.418982 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42653
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:55.519172 18144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:55.519378 18144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:55.519400 18144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:55.520512 18144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:55.520553 18144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:55.520577 18144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:55.522040 18144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:55.522289 18144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:55.522503 18144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:55.527207 18156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:55.527266 18153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:55.527400 18144 server_base.cc:1034] running on GCE node
W20260806 03:51:55.527473 18150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:55.527678 18144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:55.527910 18144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:55.529043 18144 hybrid_clock.cc:648] HybridClock initialized: now 1785988315529032 us; error 26 us; skew 500 ppm
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:55.531486 18144 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:51:55.531858 18144 webserver.cc:460] Webserver started at http://127.12.15.65:37865/ using document root <none> and password file <none>
I20260806 03:51:55.532021 18144 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:55.532059 18144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:55.532148 18144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:55.533066 18144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "73c828f2a51542aebc17554c88454168"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "dbfd569be9cefc45d2575ca3f00d506b"
server_key_iv: "5e49b1aa65859140cfcea7fdbc6e78da"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.533353 18144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "73c828f2a51542aebc17554c88454168"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "dbfd569be9cefc45d2575ca3f00d506b"
server_key_iv: "5e49b1aa65859140cfcea7fdbc6e78da"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.534536 18144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:51:55.535172 18168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.535310 18144 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:55.535386 18144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "73c828f2a51542aebc17554c88454168"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "dbfd569be9cefc45d2575ca3f00d506b"
server_key_iv: "5e49b1aa65859140cfcea7fdbc6e78da"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.535449 18144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:55.545313 18144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:55.546938 18144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:55.547058 18144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:55.547245 18144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:55.547508 18144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:55.547538 18144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.547559 18144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:55.547580 18144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.552920 18144 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:37519
I20260806 03:51:55.552958 18344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:37519 every 8 connection(s)
I20260806 03:51:55.553231 18144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:51:55.554296 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 18144
I20260806 03:51:55.554384 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:55.554530 12349 external_mini_cluster.cc:1428] Setting key f1d77cb1c3e4d66ff87d7689da277a41
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:55.562954 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.553884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:57411 (local address 127.12.15.126:42653)
0806 03:51:55.553926 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:55.553928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:55.554112 (+ 184us) server_negotiation.cc:407] Connection header received
0806 03:51:55.555224 (+ 1112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:55.555225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:55.555235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:55.555287 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:55.555866 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.556515 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.557848 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.558049 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.558094 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.560300 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:55.560302 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:55.560302 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:55.560310 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:55.561709 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.562061 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.562062 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.562063 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.562146 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.562359 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.562360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.562361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.562495 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:55.562559 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:55.562774 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 03:51:55.562860 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:51:55.563000 18347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.553850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42653 (local address 127.12.15.65:57411)
0806 03:51:55.553996 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:55.554032 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:55.555123 (+ 1091us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:55.555357 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:55.555360 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:55.555599 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:55.555782 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:55.555786 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.556596 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.556597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:55.557763 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:55.557766 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.558143 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.558145 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:55.558280 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.558727 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:55.558735 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:55.560204 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:55.561783 (+ 1579us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:55.561790 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:55.561792 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:55.561997 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:55.562206 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:55.562206 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:55.562207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:55.562305 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:55.562555 (+ 250us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:55.562558 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:55.562663 (+ 105us) client_negotiation.cc:769] Sending connection context
0806 03:51:55.562735 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:51:55.562848 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
I20260806 03:51:55.563601 18345 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42653
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
I20260806 03:51:55.563699 18345 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:55.563869 18345 heartbeater.cc:511] Master 127.12.15.126:42653 requested a full tablet report, sending...
I20260806 03:51:55.564365 18055 ts_manager.cc:194] Registered new tserver with Master: 73c828f2a51542aebc17554c88454168 (127.12.15.65:37519)
I20260806 03:51:55.564987 18055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:57411
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:51:55.579034 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42653
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:55.657928 18350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:55.658159 18350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:55.658191 18350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:55.659312 18350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:55.659353 18350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:55.659387 18350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:55.660838 18350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18350
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:55.661062 18350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:55.661275 18350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:55.665910 18359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:55.665921 18360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:55.665940 18362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:55.666193 18350 server_base.cc:1034] running on GCE node
I20260806 03:51:55.666409 18350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:55.666618 18350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:55.667749 18350 hybrid_clock.cc:648] HybridClock initialized: now 1785988315667740 us; error 32 us; skew 500 ppm
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:55.670162 18350 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:51:55.670567 18350 webserver.cc:460] Webserver started at http://127.12.15.66:34811/ using document root <none> and password file <none>
I20260806 03:51:55.670764 18350 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:55.670811 18350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:55.670936 18350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:55.671913 18350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "c2eb673a7fce47ddb017332b9a27d9b9"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "37c388518fdd7f61b77bff46d92b7064"
server_key_iv: "f825c82fcf34e9d69b9b1659db99aae7"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.672238 18350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "c2eb673a7fce47ddb017332b9a27d9b9"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "37c388518fdd7f61b77bff46d92b7064"
server_key_iv: "f825c82fcf34e9d69b9b1659db99aae7"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.673461 18350 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:55.674088 18372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.674244 18350 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.674328 18350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "c2eb673a7fce47ddb017332b9a27d9b9"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "37c388518fdd7f61b77bff46d92b7064"
server_key_iv: "f825c82fcf34e9d69b9b1659db99aae7"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.674392 18350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:55.694098 18350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:55.695814 18350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:55.695937 18350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:55.696146 18350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:55.696470 18350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:55.696514 18350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.696579 18350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:55.696609 18350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.703693 18350 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:35095
I20260806 03:51:55.703766 18542 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:35095 every 8 connection(s)
I20260806 03:51:55.704164 18350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:55.713347 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 18350
I20260806 03:51:55.713464 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:55.713667 12349 external_mini_cluster.cc:1428] Setting key 1de9a27ba5f7554b9d51d56cf3015a4e
I20260806 03:51:55.719304 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.706645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:39829 (local address 127.12.15.126:42653)
0806 03:51:55.706706 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:55.706709 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:55.706866 (+ 157us) server_negotiation.cc:407] Connection header received
0806 03:51:55.707944 (+ 1078us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:55.707946 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:55.707959 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:55.708004 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:55.708853 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.709444 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.710463 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.710663 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.710719 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.713192 (+ 2473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:55.713193 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:55.713194 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:55.713202 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:55.717662 (+ 4460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.718214 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.718215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.718215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.718279 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.718565 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.718566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.718567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.718723 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:55.718804 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:55.719116 (+ 312us) server_negotiation.cc:299] Negotiation successful
0806 03:51:55.719207 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:51:55.719458 18546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.706588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42653 (local address 127.12.15.66:39829)
0806 03:51:55.706749 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:55.706788 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:55.707828 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:55.708130 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:55.708134 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:55.708463 (+ 329us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:55.708739 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:55.708747 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.709536 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.709537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:55.710389 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:55.710391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.710732 (+ 341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.710733 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:55.710854 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.711329 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:55.711339 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:55.713066 (+ 1727us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:55.717784 (+ 4718us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:55.717792 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:55.717794 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:55.718091 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:55.718343 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:55.718344 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:55.718344 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:55.718486 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:55.718838 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:55.718846 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:55.718969 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 03:51:55.719078 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 03:51:55.719237 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":92,"threads_started":1}
I20260806 03:51:55.720050 18543 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42653
I20260806 03:51:55.720156 18543 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:55.720352 18543 heartbeater.cc:511] Master 127.12.15.126:42653 requested a full tablet report, sending...
I20260806 03:51:55.720763 18055 ts_manager.cc:194] Registered new tserver with Master: c2eb673a7fce47ddb017332b9a27d9b9 (127.12.15.66:35095)
I20260806 03:51:55.721146 18055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:39829
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:51:55.740496 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42653
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:55.819497 18549 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:55.819700 18549 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:55.819721 18549 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:55.820801 18549 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:55.820832 18549 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:55.820856 18549 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:55.822281 18549 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18549
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:55.822474 18549 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:55.822646 18549 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:55.827255 18558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:55.827311 18560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:55.827315 18557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:55.827497 18549 server_base.cc:1034] running on GCE node
I20260806 03:51:55.827702 18549 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:55.827907 18549 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:55.829039 18549 hybrid_clock.cc:648] HybridClock initialized: now 1785988315829023 us; error 33 us; skew 500 ppm
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:55.831439 18549 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:51:55.831867 18549 webserver.cc:460] Webserver started at http://127.12.15.67:37293/ using document root <none> and password file <none>
I20260806 03:51:55.832055 18549 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:55.832101 18549 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:55.832226 18549 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:55.833158 18549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "fc1a06b09b064a0b9a1a0b2f2a9def0a"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "9ab75dccbe7038329246ce84e90946f2"
server_key_iv: "f7401c8478cb0129f324c7359b50cdda"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.833482 18549 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "fc1a06b09b064a0b9a1a0b2f2a9def0a"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "9ab75dccbe7038329246ce84e90946f2"
server_key_iv: "f7401c8478cb0129f324c7359b50cdda"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.834759 18549 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:55.835495 18569 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.835665 18549 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:51:55.835752 18549 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "fc1a06b09b064a0b9a1a0b2f2a9def0a"
format_stamp: "Formatted at 2026-08-06 03:51:55 on dist-test-slave-bqcl"
server_key: "9ab75dccbe7038329246ce84e90946f2"
server_key_iv: "f7401c8478cb0129f324c7359b50cdda"
server_key_version: "encryptionkey@0"
I20260806 03:51:55.835809 18549 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:55.855139 18549 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:55.856961 18549 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:55.857090 18549 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:55.857311 18549 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:55.857589 18549 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:55.857620 18549 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.857651 18549 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:55.857674 18549 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:55.863116 18549 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:39243
I20260806 03:51:55.863170 18741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:39243 every 8 connection(s)
I20260806 03:51:55.863451 18549 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:51:55.864830 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 18549
I20260806 03:51:55.864918 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:55.865078 12349 external_mini_cluster.cc:1428] Setting key b09d77e6945a1218b86ce4aec3236cd8
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:55.872025 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.864089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:60569 (local address 127.12.15.126:42653)
0806 03:51:55.864141 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:55.864144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:55.864320 (+ 176us) server_negotiation.cc:407] Connection header received
0806 03:51:55.865196 (+ 876us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:55.865197 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:55.865210 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:55.865282 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:55.865981 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.866552 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.867467 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.867658 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.867704 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.869532 (+ 1828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:55.869533 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:55.869534 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:55.869540 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:55.870820 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.871164 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.871166 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.871166 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.871232 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.871439 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.871440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.871440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.871554 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:55.871608 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:55.871855 (+ 247us) server_negotiation.cc:299] Negotiation successful
0806 03:51:55.871935 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:51:55.872174 18747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.864056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42653 (local address 127.12.15.67:60569)
0806 03:51:55.864210 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:55.864250 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:55.865100 (+ 850us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:55.865354 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:55.865357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:55.865694 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:55.865899 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:55.865905 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.866616 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.866617 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:55.867408 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:55.867410 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.867712 (+ 302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.867713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:55.867802 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.868163 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:55.868171 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:55.869444 (+ 1273us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:55.870887 (+ 1443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:55.870892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:55.870894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:55.871094 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:55.871287 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:55.871288 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:55.871288 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:55.871385 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:55.871607 (+ 222us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:55.871613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:55.871741 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 03:51:55.871827 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 03:51:55.871971 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":64,"threads_started":1}
I20260806 03:51:55.872556 18744 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42653
I20260806 03:51:55.872648 18744 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:55.872809 18744 heartbeater.cc:511] Master 127.12.15.126:42653 requested a full tablet report, sending...
I20260806 03:51:55.873126 18055 ts_manager.cc:194] Registered new tserver with Master: fc1a06b09b064a0b9a1a0b2f2a9def0a (127.12.15.67:39243)
I20260806 03:51:55.873448 18055 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:60569
I20260806 03:51:55.876510 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:51:55.899196 18755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.892086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60448 (local address 127.12.15.65:37519)
0806 03:51:55.892277 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:55.892282 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:55.892291 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:55.892315 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:55.892316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:55.892329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:55.892388 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:55.892877 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.893563 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.894290 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.894635 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.894689 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.896603 (+ 1914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:55.896611 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:55.896611 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:55.896618 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:55.898287 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.898624 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.898626 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.898627 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.898677 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.898794 (+ 117us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.898794 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.898794 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.898902 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:55.898993 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:55.899045 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 03:51:55.899101 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":105,"threads_started":1}
Aug 06 03:51:55 dist-test-slave-bqcl krb5kdc[17986](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988315, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:55.906576 18113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:55.900365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42496 (local address 127.12.15.126:42653)
0806 03:51:55.900418 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:55.900420 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:55.900428 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:55.900494 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:55.900494 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:55.900504 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:55.900546 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:55.900971 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.901433 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.902061 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:55.902341 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:55.902396 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:55.904320 (+ 1924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:55.904321 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:55.904322 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:55.904329 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:55.905406 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.905755 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.905756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.905756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.905817 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:55.905996 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:55.905997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:55.905997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:55.906185 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:55.906288 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:55.906414 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 03:51:55.906485 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
W20260806 03:51:55.906810 18055 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42496
I20260806 03:51:55.907109 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 18144
I20260806 03:51:55.920763 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 18350
I20260806 03:51:55.933014 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 18549
I20260806 03:51:55.943286 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 18004
2026-08-06T03:51:55Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1942 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:56 dist-test-slave-bqcl krb5kdc[18763](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:56 dist-test-slave-bqcl krb5kdc[18763](info): set up 2 sockets
Aug 06 03:51:56 dist-test-slave-bqcl krb5kdc[18763](info): commencing operation
krb5kdc: starting...
W20260806 03:51:57.077718 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:51:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:51:57.191975 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36497
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:36497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:57.271615 18778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:57.271812 18778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:57.271833 18778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:57.272910 18778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:57.272943 18778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:57.272955 18778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:57.272965 18778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:57.272984 18778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:57.274766 18778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:36497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18778
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:57.274966 18778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:57.275158 18778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:57.279786 18786 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:57.279771 18792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:57.279771 18789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:57.281859 18778 server_base.cc:1034] running on GCE node
I20260806 03:51:57.282109 18778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:57.282438 18778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:57.283608 18778 hybrid_clock.cc:648] HybridClock initialized: now 1785988317283583 us; error 38 us; skew 500 ppm
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:57.288296 18778 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:51:57.288648 18778 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:57.288703 18778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:57.288830 18778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:57.289839 18778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "2872bd7a0c0947e88719e643ebde2dc9"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "9e6089e1ea4735bd27df908a90d826f3"
server_key_iv: "a29d1aa2a42db89d69438c41c46901ba"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.290220 18778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "2872bd7a0c0947e88719e643ebde2dc9"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "9e6089e1ea4735bd27df908a90d826f3"
server_key_iv: "a29d1aa2a42db89d69438c41c46901ba"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.291399 18778 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:57.292033 18802 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.292181 18778 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:57.292263 18778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "2872bd7a0c0947e88719e643ebde2dc9"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "9e6089e1ea4735bd27df908a90d826f3"
server_key_iv: "a29d1aa2a42db89d69438c41c46901ba"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.292330 18778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:57.304535 18778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:57.306226 18778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:57.306357 18778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:57.310179 18778 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:36497
I20260806 03:51:57.310206 18874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:36497 every 8 connection(s)
I20260806 03:51:57.310461 18778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:57.311512 18875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:57.313707 18875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9: Bootstrap starting.
I20260806 03:51:57.314335 18875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:57.314589 18875 log.cc:824] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:57.315168 18875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9: No bootstrap required, opened a new log
I20260806 03:51:57.316303 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 18778
I20260806 03:51:57.316404 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:57.316584 12349 external_mini_cluster.cc:1428] Setting key b44aa3cbc06d1f970df5baa0baf20cd9
I20260806 03:51:57.316732 18875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872bd7a0c0947e88719e643ebde2dc9" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36497 } }
I20260806 03:51:57.316907 18875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:57.316958 18875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2872bd7a0c0947e88719e643ebde2dc9, State: Initialized, Role: FOLLOWER
I20260806 03:51:57.317104 18875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872bd7a0c0947e88719e643ebde2dc9" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36497 } }
I20260806 03:51:57.317190 18875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:57.317224 18875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:57.317270 18875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:57.317764 18875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872bd7a0c0947e88719e643ebde2dc9" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36497 } }
I20260806 03:51:57.317854 18875 leader_election.cc:302] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2872bd7a0c0947e88719e643ebde2dc9; no voters:
I20260806 03:51:57.318010 18875 leader_election.cc:288] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:57.318051 18883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:57.318230 18883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [term 1 LEADER]: Becoming Leader. State: Replica: 2872bd7a0c0947e88719e643ebde2dc9, State: Running, Role: LEADER
I20260806 03:51:57.318287 18875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:57.318320 18883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872bd7a0c0947e88719e643ebde2dc9" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36497 } }
I20260806 03:51:57.318600 18883 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2872bd7a0c0947e88719e643ebde2dc9. Latest consensus state: current_term: 1 leader_uuid: "2872bd7a0c0947e88719e643ebde2dc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872bd7a0c0947e88719e643ebde2dc9" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36497 } } }
I20260806 03:51:57.318678 18883 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:57.318625 18885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2872bd7a0c0947e88719e643ebde2dc9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2872bd7a0c0947e88719e643ebde2dc9" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36497 } } }
I20260806 03:51:57.318825 18885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:57.319515 18892 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:57.319994 18892 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:57.321421 18892 catalog_manager.cc:1347] Generated new cluster ID: e4f620ab85f8456695510178d29d086b
I20260806 03:51:57.321471 18892 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:57.328577 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.317148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42904 (local address 127.12.15.126:36497)
0806 03:51:57.317330 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:57.317339 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:57.317365 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:51:57.317704 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:57.317709 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:57.317754 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:57.317884 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:57.318687 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.320077 (+ 1390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.320879 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.321455 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.321584 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.324420 (+ 2836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:57.324424 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:57.324425 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:57.324444 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:57.326960 (+ 2516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.327332 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.327335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.327336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.327447 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.327624 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.327624 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.327625 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.327954 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:57.328131 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:57.328204 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 03:51:57.328344 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":68,"threads_started":1}
I20260806 03:51:57.336634 18892 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:57.337368 18892 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:57.345417 18892 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9: Generated new TSK 0
I20260806 03:51:57.345630 18892 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:51:57.373461 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36497
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:57.453565 18913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:57.453750 18913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:57.453781 18913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:57.454921 18913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:57.454962 18913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:57.454995 18913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:57.456437 18913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:57.456658 18913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:57.456866 18913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:57.461465 18925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:57.461467 18927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:57.461467 18923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:57.461814 18913 server_base.cc:1034] running on GCE node
I20260806 03:51:57.461989 18913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:57.462257 18913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:57.463408 18913 hybrid_clock.cc:648] HybridClock initialized: now 1785988317463388 us; error 35 us; skew 500 ppm
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:57.465888 18913 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:51:57.466256 18913 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:57.466312 18913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:57.466449 18913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:57.467464 18913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "e7d246752cafbedc2effb50e3be475ef"
server_key_iv: "57db5fcef74467ee9e46f2b378490e42"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.467796 18913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "e7d246752cafbedc2effb50e3be475ef"
server_key_iv: "57db5fcef74467ee9e46f2b378490e42"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.468979 18913 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:57.469652 18937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.469812 18913 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:57.469895 18913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "e7d246752cafbedc2effb50e3be475ef"
server_key_iv: "57db5fcef74467ee9e46f2b378490e42"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.469964 18913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:57.495157 18913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:57.496959 18913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:57.497092 18913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:57.497301 18913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:57.497606 18913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:57.497640 18913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.497669 18913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:57.497686 18913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.503237 18913 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:43691
I20260806 03:51:57.503295 19120 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:43691 every 8 connection(s)
I20260806 03:51:57.503527 18913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:57.509073 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 18913
I20260806 03:51:57.509192 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:57.509372 12349 external_mini_cluster.cc:1428] Setting key cdf86c5f068594f604d59f2411ce5fc5
I20260806 03:51:57.512239 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.504150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:47841 (local address 127.12.15.126:36497)
0806 03:51:57.504206 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:57.504208 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:57.504407 (+ 199us) server_negotiation.cc:407] Connection header received
0806 03:51:57.504862 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:57.504863 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:57.504875 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:57.504926 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:57.505920 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.506425 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.507357 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.507543 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.507586 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.509559 (+ 1973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:57.509560 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:57.509561 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:57.509569 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:57.510891 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.511322 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.511323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.511323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.511366 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.511628 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.511629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.511630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.511774 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:57.511843 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:57.512063 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 03:51:57.512144 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:51:57.512317 19123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.504129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36497 (local address 127.12.15.65:47841)
0806 03:51:57.504283 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:57.504324 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:57.504787 (+ 463us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:57.504982 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:57.504986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:57.505244 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:57.505848 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.505854 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.506485 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.506486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.507294 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.507297 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.507596 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.507596 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.507688 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.508079 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:57.508087 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:57.509458 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:57.510967 (+ 1509us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:57.510975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:57.510977 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:57.511256 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:57.511413 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:57.511414 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:57.511414 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:57.511544 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:57.511836 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:57.511841 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:57.511951 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 03:51:57.512031 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:51:57.512154 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":65,"threads_started":1}
I20260806 03:51:57.512979 19121 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36497
I20260806 03:51:57.513074 19121 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:57.513213 19121 heartbeater.cc:511] Master 127.12.15.126:36497 requested a full tablet report, sending...
I20260806 03:51:57.513706 18828 ts_manager.cc:194] Registered new tserver with Master: bff4ae4ff1d04f84b2fafd0d43b706d7 (127.12.15.65:43691)
I20260806 03:51:57.514418 18828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:47841
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:51:57.532670 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36497
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:57.611572 19127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:57.611775 19127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:57.611799 19127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:57.612907 19127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:57.612947 19127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:57.612977 19127 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:57.614440 19127 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:57.614665 19127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:57.614872 19127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:57.619875 19136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:57.619920 19137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:57.619875 19142 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:57.620240 19127 server_base.cc:1034] running on GCE node
I20260806 03:51:57.620465 19127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:57.620697 19127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:57.621837 19127 hybrid_clock.cc:648] HybridClock initialized: now 1785988317621825 us; error 25 us; skew 500 ppm
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:57.624290 19127 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:51:57.624640 19127 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:57.624707 19127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:57.624840 19127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:57.625839 19127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "c50ae0d73bb64050935b606f3e35cd2d"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "0acb19837bb84a4f3097e550ff7c9c94"
server_key_iv: "a2b9edfa26059788cd679ec374ef7da6"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.626211 19127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "c50ae0d73bb64050935b606f3e35cd2d"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "0acb19837bb84a4f3097e550ff7c9c94"
server_key_iv: "a2b9edfa26059788cd679ec374ef7da6"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.627410 19127 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:57.628098 19149 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.628259 19127 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:57.628341 19127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "c50ae0d73bb64050935b606f3e35cd2d"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "0acb19837bb84a4f3097e550ff7c9c94"
server_key_iv: "a2b9edfa26059788cd679ec374ef7da6"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.628407 19127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:57.639775 19127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:57.641778 19127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:57.641906 19127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:57.642091 19127 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:57.642392 19127 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:57.642426 19127 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.642453 19127 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:57.642477 19127 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.648028 19127 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:39083
I20260806 03:51:57.648084 19337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:39083 every 8 connection(s)
I20260806 03:51:57.648310 19127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:57.656280 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.648884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:57835 (local address 127.12.15.126:36497)
0806 03:51:57.648967 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:57.648969 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:57.649152 (+ 183us) server_negotiation.cc:407] Connection header received
0806 03:51:57.649610 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:57.649612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:57.649622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:57.649667 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:57.650553 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.651048 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.651964 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.652134 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.652176 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.654017 (+ 1841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:57.654018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:57.654018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:57.654026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:57.655279 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.655610 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.655611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.655611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.655655 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.655820 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.655820 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.655821 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.655910 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:57.655952 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:57.656155 (+ 203us) server_negotiation.cc:299] Negotiation successful
0806 03:51:57.656217 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 03:51:57.656447 19340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.648863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36497 (local address 127.12.15.66:57835)
0806 03:51:57.649015 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:57.649059 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:57.649523 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:57.649728 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:57.649731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:57.649973 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:57.650481 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.650487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.651109 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.651110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.651903 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.651905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.652184 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.652184 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.652276 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.652644 (+ 368us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:57.652652 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:57.653935 (+ 1283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:57.655349 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:57.655354 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:57.655355 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:57.655559 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:57.655690 (+ 131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:57.655691 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:57.655691 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:57.655781 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:57.655950 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:57.655952 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:57.656065 (+ 113us) client_negotiation.cc:769] Sending connection context
0806 03:51:57.656140 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:51:57.656251 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260806 03:51:57.656880 19338 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36497
I20260806 03:51:57.656966 19338 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:57.657122 19338 heartbeater.cc:511] Master 127.12.15.126:36497 requested a full tablet report, sending...
I20260806 03:51:57.657297 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 19127
I20260806 03:51:57.657398 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:57.657506 18828 ts_manager.cc:194] Registered new tserver with Master: c50ae0d73bb64050935b606f3e35cd2d (127.12.15.66:39083)
I20260806 03:51:57.657590 12349 external_mini_cluster.cc:1428] Setting key 20e133a9519260651abdcf7ad556b6be
I20260806 03:51:57.657922 18828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:57835
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:51:57.680550 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36497
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41245
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:51:57.759159 19346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:57.759349 19346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:57.759370 19346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:57.760425 19346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:57.760458 19346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:57.760483 19346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:57.761900 19346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41245
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36497
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:57.762082 19346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:57.762295 19346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:57.766870 19357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:57.766884 19356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:57.766877 19362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:57.767114 19346 server_base.cc:1034] running on GCE node
I20260806 03:51:57.767318 19346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:57.767540 19346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:57.768688 19346 hybrid_clock.cc:648] HybridClock initialized: now 1785988317768673 us; error 29 us; skew 500 ppm
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:51:57.771040 19346 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:51:57.771376 19346 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:57.771425 19346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:57.771512 19346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:57.772475 19346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "f04808d922484d598559822f88b3c47b"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "5836f042199503e2eb7d05193291d121"
server_key_iv: "11f85224348f2cd4150d721274a9dc02"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.772768 19346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "f04808d922484d598559822f88b3c47b"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "5836f042199503e2eb7d05193291d121"
server_key_iv: "11f85224348f2cd4150d721274a9dc02"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.773895 19346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:57.774595 19371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.774735 19346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.774817 19346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "f04808d922484d598559822f88b3c47b"
format_stamp: "Formatted at 2026-08-06 03:51:57 on dist-test-slave-bqcl"
server_key: "5836f042199503e2eb7d05193291d121"
server_key_iv: "11f85224348f2cd4150d721274a9dc02"
server_key_version: "encryptionkey@0"
I20260806 03:51:57.774875 19346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:57.785374 19346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:57.786999 19346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:57.787127 19346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:57.787310 19346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:57.787590 19346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:57.787623 19346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.787650 19346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:57.787667 19346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:57.793351 19346 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:35761
I20260806 03:51:57.793390 19543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:35761 every 8 connection(s)
I20260806 03:51:57.793634 19346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:51:57.794641 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 19346
I20260806 03:51:57.794744 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:57.794903 12349 external_mini_cluster.cc:1428] Setting key 721cda6833bf29c8c1572f3318bbfb0b
Aug 06 03:51:57 dist-test-slave-bqcl krb5kdc[18763](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988317, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:51:57.801934 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.794305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:42925 (local address 127.12.15.126:36497)
0806 03:51:57.794359 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:57.794361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:57.794583 (+ 222us) server_negotiation.cc:407] Connection header received
0806 03:51:57.795122 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:57.795124 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:57.795137 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:57.795207 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:57.796217 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.796723 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.797618 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.797798 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.797843 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.799839 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:57.799840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:57.799840 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:57.799847 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:57.800920 (+ 1073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.801239 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.801239 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.801240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.801282 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.801458 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.801459 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.801459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.801569 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:57.801615 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:57.801812 (+ 197us) server_negotiation.cc:299] Negotiation successful
0806 03:51:57.801869 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:51:57.802076 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.794318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36497 (local address 127.12.15.67:42925)
0806 03:51:57.794456 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:57.794494 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:57.795017 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:57.795261 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:57.795265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:57.795602 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:57.796135 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.796141 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.796782 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.796783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.797548 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.797551 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.797850 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.797851 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.797941 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.798425 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:51:57.798433 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:57.799759 (+ 1326us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:57.800982 (+ 1223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:57.800988 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:57.800990 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:57.801184 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:57.801324 (+ 140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:51:57.801324 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:51:57.801324 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:51:57.801419 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:51:57.801610 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:57.801613 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:57.801721 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 03:51:57.801794 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:51:57.801901 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":69,"threads_started":1}
I20260806 03:51:57.802516 19546 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36497
I20260806 03:51:57.802598 19546 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:57.802728 19546 heartbeater.cc:511] Master 127.12.15.126:36497 requested a full tablet report, sending...
I20260806 03:51:57.803052 18827 ts_manager.cc:194] Registered new tserver with Master: f04808d922484d598559822f88b3c47b (127.12.15.67:35761)
I20260806 03:51:57.803406 18827 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:42925
I20260806 03:51:57.806569 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:51:57.812546 18882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.807766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42910 (local address 127.12.15.126:36497)
0806 03:51:57.807805 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:57.807807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:57.807899 (+ 92us) server_negotiation.cc:407] Connection header received
0806 03:51:57.807947 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:57.807947 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:57.807954 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:57.808002 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:57.808491 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.808945 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.809580 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.809744 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.809784 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.810771 (+ 987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:57.810772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:57.810772 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:51:57.810776 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:57.811765 (+ 989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.812026 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.812027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.812027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.812065 (+ 38us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:51:57.812168 (+ 103us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:51:57.812169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:51:57.812169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:51:57.812282 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:57.812346 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:57.812420 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 03:51:57.812471 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:51:57.813752 18827 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42910:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:51:57.815322 18827 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:57.821540 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.817307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60016 (local address 127.12.15.65:43691)
0806 03:51:57.817508 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:57.817514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:57.817657 (+ 143us) server_negotiation.cc:407] Connection header received
0806 03:51:57.817749 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:57.817752 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:57.817768 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:57.817870 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:57.818112 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.819349 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.820429 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.821146 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.821228 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.821291 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:57.821416 (+ 125us) server_negotiation.cc:299] Negotiation successful
0806 03:51:57.821458 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":73,"threads_started":1}
I20260806 03:51:57.821543 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.817391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:35761 (local address 127.0.0.1:52380)
0806 03:51:57.817735 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:57.817741 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:57.817778 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:57.817992 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:57.817993 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:57.817995 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:57.818029 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.818030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.819338 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.819339 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.820472 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.820475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.821255 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.821255 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.821310 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.821324 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:51:57.821397 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:51:57.821455 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":46,"threads_started":1}
I20260806 03:51:57.821544 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.817203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:43691 (local address 127.0.0.1:60016)
0806 03:51:57.817475 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:57.817516 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:57.817581 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:57.817881 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:57.817883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:57.817885 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:57.818029 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.818031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.819434 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.819435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.820353 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.820356 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.821225 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.821226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.821310 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.821324 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:51:57.821397 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:51:57.821455 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"mutex_wait_us":21,"thread_start_us":48,"threads_started":1}
I20260806 03:51:57.821914 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.817479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.12.15.67:35761)
0806 03:51:57.817604 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:57.817608 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:57.817790 (+ 182us) server_negotiation.cc:407] Connection header received
0806 03:51:57.817830 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:57.817832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:57.817846 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:57.817916 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:57.818107 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.819246 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.820552 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.821185 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.821245 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.821291 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:57.821806 (+ 515us) server_negotiation.cc:299] Negotiation successful
0806 03:51:57.821842 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":69,"threads_started":1}
I20260806 03:51:57.822408 19013 tablet_service.cc:1467] Processing CreateTablet for tablet 6edecb36b2964bd59cb857485ed09c0c (DEFAULT_TABLE table=test-table [id=82e3d0acd1584050a9f17a524a24ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:57.822584 19558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.817207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:39083 (local address 127.0.0.1:36044)
0806 03:51:57.817484 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:57.817521 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:57.817579 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:57.817914 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:57.817915 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:57.817917 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:57.818035 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.818037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.819137 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.819138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.820473 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:57.820475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.822346 (+ 1871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.822347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:57.822415 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.822418 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:57.822467 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:51:57.822506 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"mutex_wait_us":25,"thread_start_us":61,"threads_started":1}
I20260806 03:51:57.822613 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:57.817322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36044 (local address 127.12.15.66:39083)
0806 03:51:57.817689 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:57.817693 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:57.817703 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:51:57.817740 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:57.817742 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:57.817757 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:57.817830 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:57.818116 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.819046 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.821327 (+ 2281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:57.822197 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:57.822256 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:57.822313 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:57.822482 (+ 169us) server_negotiation.cc:299] Negotiation successful
0806 03:51:57.822541 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":71,"threads_started":1}
I20260806 03:51:57.822774 19013 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6edecb36b2964bd59cb857485ed09c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:57.825201 19569 tablet_bootstrap.cc:492] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7: Bootstrap starting.
I20260806 03:51:57.825626 19438 tablet_service.cc:1467] Processing CreateTablet for tablet 6edecb36b2964bd59cb857485ed09c0c (DEFAULT_TABLE table=test-table [id=82e3d0acd1584050a9f17a524a24ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:57.825874 19569 tablet_bootstrap.cc:654] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:57.825937 19438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6edecb36b2964bd59cb857485ed09c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:57.826561 19569 log.cc:824] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:57.826615 19229 tablet_service.cc:1467] Processing CreateTablet for tablet 6edecb36b2964bd59cb857485ed09c0c (DEFAULT_TABLE table=test-table [id=82e3d0acd1584050a9f17a524a24ccdd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:57.826910 19229 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6edecb36b2964bd59cb857485ed09c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:57.828615 19574 tablet_bootstrap.cc:492] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Bootstrap starting.
I20260806 03:51:57.829000 19569 tablet_bootstrap.cc:492] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7: No bootstrap required, opened a new log
I20260806 03:51:57.829071 19569 ts_tablet_manager.cc:1397] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260806 03:51:57.829246 19574 tablet_bootstrap.cc:654] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:57.829522 19577 tablet_bootstrap.cc:492] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Bootstrap starting.
I20260806 03:51:57.829803 19574 log.cc:824] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:57.830084 19577 tablet_bootstrap.cc:654] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:57.830641 19577 log.cc:824] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:57.830646 19574 tablet_bootstrap.cc:492] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: No bootstrap required, opened a new log
I20260806 03:51:57.830717 19574 ts_tablet_manager.cc:1397] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:57.831229 19577 tablet_bootstrap.cc:492] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: No bootstrap required, opened a new log
I20260806 03:51:57.831091 19569 raft_consensus.cc:348] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:57.831275 19569 raft_consensus.cc:374] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:57.831288 19577 ts_tablet_manager.cc:1397] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:51:57.831321 19569 raft_consensus.cc:729] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bff4ae4ff1d04f84b2fafd0d43b706d7, State: Initialized, Role: FOLLOWER
I20260806 03:51:57.831434 19569 consensus_queue.cc:260] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:57.831707 19569 ts_tablet_manager.cc:1428] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:51:57.831842 19121 heartbeater.cc:503] Master 127.12.15.126:36497 was elected leader, sending a full tablet report...
I20260806 03:51:57.832608 19574 raft_consensus.cc:348] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:57.832769 19574 raft_consensus.cc:374] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:57.832815 19574 raft_consensus.cc:729] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f04808d922484d598559822f88b3c47b, State: Initialized, Role: FOLLOWER
I20260806 03:51:57.832926 19574 consensus_queue.cc:260] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:57.833189 19574 ts_tablet_manager.cc:1428] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:57.833216 19546 heartbeater.cc:503] Master 127.12.15.126:36497 was elected leader, sending a full tablet report...
I20260806 03:51:57.833262 19577 raft_consensus.cc:348] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:57.833403 19577 raft_consensus.cc:374] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:57.833444 19577 raft_consensus.cc:729] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c50ae0d73bb64050935b606f3e35cd2d, State: Initialized, Role: FOLLOWER
I20260806 03:51:57.833550 19577 consensus_queue.cc:260] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:57.833825 19577 ts_tablet_manager.cc:1428] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260806 03:51:57.833880 19338 heartbeater.cc:503] Master 127.12.15.126:36497 was elected leader, sending a full tablet report...
W20260806 03:51:57.898793 19339 tablet.cc:2369] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:51:58.004256 19122 tablet.cc:2369] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:51:58.044166 19549 tablet.cc:2369] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:51:58.077250 19580 raft_consensus.cc:482] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:58.077382 19580 raft_consensus.cc:504] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:58.077812 19580 leader_election.cc:288] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c50ae0d73bb64050935b606f3e35cd2d (127.12.15.66:39083), f04808d922484d598559822f88b3c47b (127.12.15.67:35761)
I20260806 03:51:58.080593 19567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.078148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:45895 (local address 127.12.15.67:35761)
0806 03:51:58.078210 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.078212 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.078221 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:51:58.078292 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.078293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.078302 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.078353 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:58.078516 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.078988 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.079928 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.080295 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.080357 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:58.080366 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.080486 (+ 120us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.080529 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:51:58.080611 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.078077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:58379 (local address 127.12.15.66:39083)
0806 03:51:58.078155 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.078157 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.078246 (+ 89us) server_negotiation.cc:407] Connection header received
0806 03:51:58.078295 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.078296 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.078306 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.078352 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:51:58.078543 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.078998 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.079938 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.080299 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.080362 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:58.080369 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.080510 (+ 141us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.080543 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:51:58.080638 19586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.077990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:39083 (local address 127.12.15.65:58379)
0806 03:51:58.078157 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.078168 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.078239 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.078363 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.078363 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.078365 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:58.078459 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:58.078461 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.079071 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.079072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:58.079853 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:58.079855 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.080369 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.080370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:58.080447 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:58.080451 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.080495 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.080533 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":13,"thread_start_us":79,"threads_started":1}
I20260806 03:51:58.080647 19585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.077997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:35761 (local address 127.12.15.65:45895)
0806 03:51:58.078166 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.078176 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.078235 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.078360 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.078361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.078363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:51:58.078438 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:58.078452 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.079060 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.079061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:58.079850 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:58.079853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.080362 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.080363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:58.080429 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:58.080432 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.080475 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.080516 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":18,"thread_start_us":71,"threads_started":1}
I20260806 03:51:58.080945 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6edecb36b2964bd59cb857485ed09c0c" candidate_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c50ae0d73bb64050935b606f3e35cd2d" is_pre_election: true
I20260806 03:51:58.081074 19265 raft_consensus.cc:2454] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bff4ae4ff1d04f84b2fafd0d43b706d7 in term 0.
I20260806 03:51:58.081290 18939 leader_election.cc:302] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bff4ae4ff1d04f84b2fafd0d43b706d7, c50ae0d73bb64050935b606f3e35cd2d; no voters:
I20260806 03:51:58.081322 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6edecb36b2964bd59cb857485ed09c0c" candidate_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f04808d922484d598559822f88b3c47b" is_pre_election: true
I20260806 03:51:58.081452 19469 raft_consensus.cc:2454] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bff4ae4ff1d04f84b2fafd0d43b706d7 in term 0.
I20260806 03:51:58.081506 19580 raft_consensus.cc:2781] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:58.081578 19580 raft_consensus.cc:482] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:58.081609 19580 raft_consensus.cc:3037] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:58.082357 19580 raft_consensus.cc:504] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:58.082480 19580 leader_election.cc:288] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [CANDIDATE]: Term 1 election: Requested vote from peers c50ae0d73bb64050935b606f3e35cd2d (127.12.15.66:39083), f04808d922484d598559822f88b3c47b (127.12.15.67:35761)
I20260806 03:51:58.082640 19265 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6edecb36b2964bd59cb857485ed09c0c" candidate_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c50ae0d73bb64050935b606f3e35cd2d"
I20260806 03:51:58.082643 19469 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6edecb36b2964bd59cb857485ed09c0c" candidate_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f04808d922484d598559822f88b3c47b"
I20260806 03:51:58.082713 19469 raft_consensus.cc:3037] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:58.082718 19265 raft_consensus.cc:3037] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:58.083374 19265 raft_consensus.cc:2454] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff4ae4ff1d04f84b2fafd0d43b706d7 in term 1.
I20260806 03:51:58.083412 19469 raft_consensus.cc:2454] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bff4ae4ff1d04f84b2fafd0d43b706d7 in term 1.
I20260806 03:51:58.083537 18939 leader_election.cc:302] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bff4ae4ff1d04f84b2fafd0d43b706d7, c50ae0d73bb64050935b606f3e35cd2d; no voters:
I20260806 03:51:58.083621 19580 raft_consensus.cc:2781] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:58.083766 19580 raft_consensus.cc:686] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [term 1 LEADER]: Becoming Leader. State: Replica: bff4ae4ff1d04f84b2fafd0d43b706d7, State: Running, Role: LEADER
I20260806 03:51:58.083848 19580 consensus_queue.cc:237] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } }
I20260806 03:51:58.084585 18828 catalog_manager.cc:5495] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 reported cstate change: term changed from 0 to 1, leader changed from <none> to bff4ae4ff1d04f84b2fafd0d43b706d7 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bff4ae4ff1d04f84b2fafd0d43b706d7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 } health_report { overall_health: UNKNOWN } } }
I20260806 03:51:58.101050 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.099063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60026 (local address 127.12.15.65:43691)
0806 03:51:58.099113 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.099115 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.099123 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:58.099157 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.099157 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.099174 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.099229 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:51:58.099322 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.099765 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.100479 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.100670 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.100722 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:58.100806 (+ 84us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:51:58.100911 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:51:58.100964 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.100976 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.100998 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:51:58.102808 19265 raft_consensus.cc:1261] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 1 FOLLOWER]: Refusing update from remote peer bff4ae4ff1d04f84b2fafd0d43b706d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:58.102808 19469 raft_consensus.cc:1261] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 1 FOLLOWER]: Refusing update from remote peer bff4ae4ff1d04f84b2fafd0d43b706d7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:58.103137 19580 consensus_queue.cc:1035] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c50ae0d73bb64050935b606f3e35cd2d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:58.103269 19587 consensus_queue.cc:1035] T 6edecb36b2964bd59cb857485ed09c0c P bff4ae4ff1d04f84b2fafd0d43b706d7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f04808d922484d598559822f88b3c47b" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 35761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:58.105605 19592 mvcc.cc:205] Tried to move back new op lower bound from 7315408150946480128 to 7315408150872014848. Current Snapshot: MvccSnapshot[applied={T|T < 7315408150946480128}]
I20260806 03:51:58.107831 18828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:58.107914 18828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42910:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:58.108632 18828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2872bd7a0c0947e88719e643ebde2dc9: Sending DeleteTablet for 3 replicas of tablet 6edecb36b2964bd59cb857485ed09c0c
I20260806 03:51:58.108940 19229 tablet_service.cc:1514] Processing DeleteTablet for tablet 6edecb36b2964bd59cb857485ed09c0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:58 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:36044
I20260806 03:51:58.108925 19013 tablet_service.cc:1514] Processing DeleteTablet for tablet 6edecb36b2964bd59cb857485ed09c0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:58 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:60016
I20260806 03:51:58.108968 19438 tablet_service.cc:1514] Processing DeleteTablet for tablet 6edecb36b2964bd59cb857485ed09c0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:58 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:52380
I20260806 03:51:58.109184 19611 tablet_replica.cc:330] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: stopping tablet replica
I20260806 03:51:58.109284 19611 raft_consensus.cc:2229] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:58.109313 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 18913
I20260806 03:51:58.109383 19611 raft_consensus.cc:2258] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:58.111667 19607 tablet_replica.cc:330] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: stopping tablet replica
I20260806 03:51:58.111750 19607 raft_consensus.cc:2229] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:58.111824 19607 raft_consensus.cc:2258] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:58.117205 19611 ts_tablet_manager.cc:1905] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:58.117909 19607 ts_tablet_manager.cc:1905] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:58.118853 19611 ts_tablet_manager.cc:1918] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:51:58.118930 19611 log.cc:1196] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/6edecb36b2964bd59cb857485ed09c0c
I20260806 03:51:58.119184 19611 ts_tablet_manager.cc:1939] T 6edecb36b2964bd59cb857485ed09c0c P f04808d922484d598559822f88b3c47b: Deleting consensus metadata
I20260806 03:51:58.119480 19607 ts_tablet_manager.cc:1918] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:51:58.119553 19607 log.cc:1196] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/6edecb36b2964bd59cb857485ed09c0c
I20260806 03:51:58.119637 18805 catalog_manager.cc:4841] TS f04808d922484d598559822f88b3c47b (127.12.15.67:35761): tablet 6edecb36b2964bd59cb857485ed09c0c (table test-table [id=82e3d0acd1584050a9f17a524a24ccdd]) successfully deleted
I20260806 03:51:58.119799 19607 ts_tablet_manager.cc:1939] T 6edecb36b2964bd59cb857485ed09c0c P c50ae0d73bb64050935b606f3e35cd2d: Deleting consensus metadata
I20260806 03:51:58.120162 18804 catalog_manager.cc:4841] TS c50ae0d73bb64050935b606f3e35cd2d (127.12.15.66:39083): tablet 6edecb36b2964bd59cb857485ed09c0c (table test-table [id=82e3d0acd1584050a9f17a524a24ccdd]) successfully deleted
W20260806 03:51:58.124539 18805 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:43691 (error 108)
I20260806 03:51:58.124822 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 19127
I20260806 03:51:58.124955 19558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.124844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:43691 (local address 127.0.0.1:60032)
0806 03:51:58.124890 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.124923 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:43691: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 03:51:58.126211 18805 catalog_manager.cc:4584] TS bff4ae4ff1d04f84b2fafd0d43b706d7 (127.12.15.65:43691): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6edecb36b2964bd59cb857485ed09c0c: Network error: Client connection negotiation failed: client connection to 127.12.15.65:43691: connect: Connection refused (error 111)
I20260806 03:51:58.132077 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 19346
I20260806 03:51:58.138695 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 18778
2026-08-06T03:51:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2190 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-06T03:51:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:58Z Disabled control of system clock
I20260806 03:51:58.169664 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36829
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:36829
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 03:51:58.249464 19621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:58.249655 19621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:58.249675 19621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:58.250824 19621 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:58.250865 19621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:58.250877 19621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:58.250887 19621 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:58.250898 19621 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:51:58.252192 19621 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:36829
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36829
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:58.252420 19621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:58.252689 19621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:58.257324 19628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:58.257334 19630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:58.257352 19632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:58.257583 19621 server_base.cc:1034] running on GCE node
I20260806 03:51:58.257826 19621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:58.258108 19621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:58.259281 19621 hybrid_clock.cc:648] HybridClock initialized: now 1785988318259273 us; error 28 us; skew 500 ppm
I20260806 03:51:58.260504 19621 webserver.cc:460] Webserver started at http://127.12.15.126:34523/ using document root <none> and password file <none>
I20260806 03:51:58.260722 19621 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:58.260772 19621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:58.260903 19621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:58.261943 19621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "ba345a38123244d4a02a1d1316f1ee6f"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "1906d7e35a019bd064c62cfcb7bcb302"
server_key_iv: "7cb9a51b61fface4d1c44367856bf7db"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.262308 19621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "ba345a38123244d4a02a1d1316f1ee6f"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "1906d7e35a019bd064c62cfcb7bcb302"
server_key_iv: "7cb9a51b61fface4d1c44367856bf7db"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.263540 19621 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:58.264240 19641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.264413 19621 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:58.264494 19621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "ba345a38123244d4a02a1d1316f1ee6f"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "1906d7e35a019bd064c62cfcb7bcb302"
server_key_iv: "7cb9a51b61fface4d1c44367856bf7db"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.264557 19621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:58.267897 19621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:58.268137 19621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:58.268265 19621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:58.272061 19621 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:36829
I20260806 03:51:58.272111 19724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:36829 every 8 connection(s)
I20260806 03:51:58.272408 19621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:58.273078 19727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:58.273591 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 19621
I20260806 03:51:58.273676 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:58.273845 12349 external_mini_cluster.cc:1428] Setting key 332cfdc9702bb1fa4eec06d69d969928
I20260806 03:51:58.276233 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.274356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47542 (local address 127.12.15.126:36829)
0806 03:51:58.274576 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.274586 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.274618 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:51:58.275081 (+ 463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.275090 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.275145 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.275295 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.275827 (+ 532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.275831 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.275831 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.275835 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.275881 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.275894 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.275967 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.275999 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.276054 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":84,"threads_started":1}
I20260806 03:51:58.276829 19727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f: Bootstrap starting.
I20260806 03:51:58.277446 19727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:58.277709 19727 log.cc:824] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:58.278378 19727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f: No bootstrap required, opened a new log
I20260806 03:51:58.279780 19727 raft_consensus.cc:348] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba345a38123244d4a02a1d1316f1ee6f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36829 } }
I20260806 03:51:58.279914 19727 raft_consensus.cc:374] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:58.279945 19727 raft_consensus.cc:729] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ba345a38123244d4a02a1d1316f1ee6f, State: Initialized, Role: FOLLOWER
I20260806 03:51:58.280072 19727 consensus_queue.cc:260] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba345a38123244d4a02a1d1316f1ee6f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36829 } }
I20260806 03:51:58.280143 19727 raft_consensus.cc:388] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:58.280174 19727 raft_consensus.cc:482] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:58.280218 19727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:58.280731 19727 raft_consensus.cc:504] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba345a38123244d4a02a1d1316f1ee6f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36829 } }
I20260806 03:51:58.280819 19727 leader_election.cc:302] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ba345a38123244d4a02a1d1316f1ee6f; no voters:
I20260806 03:51:58.280962 19727 leader_election.cc:288] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:58.281205 19727 sys_catalog.cc:564] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:58.281209 19734 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:58.281381 19734 raft_consensus.cc:686] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [term 1 LEADER]: Becoming Leader. State: Replica: ba345a38123244d4a02a1d1316f1ee6f, State: Running, Role: LEADER
I20260806 03:51:58.281483 19734 consensus_queue.cc:237] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba345a38123244d4a02a1d1316f1ee6f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36829 } }
I20260806 03:51:58.281839 19736 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ba345a38123244d4a02a1d1316f1ee6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba345a38123244d4a02a1d1316f1ee6f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36829 } } }
I20260806 03:51:58.281941 19736 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:58.282084 19737 sys_catalog.cc:455] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ba345a38123244d4a02a1d1316f1ee6f. Latest consensus state: current_term: 1 leader_uuid: "ba345a38123244d4a02a1d1316f1ee6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ba345a38123244d4a02a1d1316f1ee6f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36829 } } }
I20260806 03:51:58.282178 19737 sys_catalog.cc:458] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:58.282145 19744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:58.282712 19744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:58.284093 19744 catalog_manager.cc:1347] Generated new cluster ID: db075882f9d147c0af5ab0beb0e8436e
I20260806 03:51:58.284147 19744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:58.293625 19744 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:58.293778 19744 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:58.300882 19744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f: Generated new TSK 0
I20260806 03:51:58.301055 19744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:58.309713 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36829
--builtin_ntp_servers=127.12.15.84:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 03:51:58.413051 19765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:58.413245 19765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:58.413266 19765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:58.414418 19765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:58.414458 19765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:58.414481 19765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:51:58.415788 19765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:58.415999 19765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:58.416290 19765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:58.421842 19772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:58.421906 19777 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:58.422009 19771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:58.422389 19765 server_base.cc:1034] running on GCE node
I20260806 03:51:58.422606 19765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:58.422888 19765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:58.424057 19765 hybrid_clock.cc:648] HybridClock initialized: now 1785988318424040 us; error 38 us; skew 500 ppm
I20260806 03:51:58.425279 19765 webserver.cc:460] Webserver started at http://127.12.15.65:34825/ using document root <none> and password file <none>
I20260806 03:51:58.425509 19765 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:58.425565 19765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:58.425690 19765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:58.426762 19765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "c72ccc3c7a8c4431a828efbd78f97a89"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "cbce1c4404c52507bf575fe69b79e89c"
server_key_iv: "d1c04ecbb7f16beff4510a0557261345"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.427101 19765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "c72ccc3c7a8c4431a828efbd78f97a89"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "cbce1c4404c52507bf575fe69b79e89c"
server_key_iv: "d1c04ecbb7f16beff4510a0557261345"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.428297 19765 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:51:58.428972 19787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.429138 19765 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:58.429234 19765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "c72ccc3c7a8c4431a828efbd78f97a89"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "cbce1c4404c52507bf575fe69b79e89c"
server_key_iv: "d1c04ecbb7f16beff4510a0557261345"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.429312 19765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:58.434063 19765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:58.434325 19765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:58.434433 19765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:58.434619 19765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:58.434882 19765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:58.434914 19765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.434942 19765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:58.434962 19765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.441092 19765 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:40345
I20260806 03:51:58.441128 19970 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:40345 every 8 connection(s)
I20260806 03:51:58.441398 19765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:51:58.444300 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.442053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:45819 (local address 127.12.15.126:36829)
0806 03:51:58.442110 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.442111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.442359 (+ 248us) server_negotiation.cc:407] Connection header received
0806 03:51:58.442829 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.442831 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.442838 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.442906 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.444037 (+ 1131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.444039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.444039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.444044 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.444060 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.444065 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.444118 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.444209 (+ 91us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.444224 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:51:58.444475 19973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.442047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36829 (local address 127.12.15.65:45819)
0806 03:51:58.442243 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.442276 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.442727 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.442983 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.442986 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.442995 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:58.443880 (+ 885us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:51:58.443881 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:58.443907 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:51:58.443935 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:58.444133 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:58.444139 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:58.444142 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.444214 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.444275 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":104,"threads_started":1}
I20260806 03:51:58.444962 19971 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36829
I20260806 03:51:58.445056 19971 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:58.445170 19971 heartbeater.cc:511] Master 127.12.15.126:36829 requested a full tablet report, sending...
I20260806 03:51:58.445631 19672 ts_manager.cc:194] Registered new tserver with Master: c72ccc3c7a8c4431a828efbd78f97a89 (127.12.15.65:40345)
I20260806 03:51:58.445824 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 19765
I20260806 03:51:58.445935 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:51:58.446173 12349 external_mini_cluster.cc:1428] Setting key e1e4366e2eef0f2d957d75ccb153c2b6
I20260806 03:51:58.447047 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36829
--builtin_ntp_servers=127.12.15.84:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 03:51:58.527415 19974 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:58.527614 19974 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:58.527634 19974 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:58.528707 19974 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:58.528739 19974 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:58.528762 19974 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:51:58.530005 19974 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19974
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:58.530251 19974 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:58.530503 19974 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:58.535158 19990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:58.535211 19993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:58.535259 19989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:58.535463 19974 server_base.cc:1034] running on GCE node
I20260806 03:51:58.535655 19974 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:58.535871 19974 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:58.537014 19974 hybrid_clock.cc:648] HybridClock initialized: now 1785988318537002 us; error 29 us; skew 500 ppm
I20260806 03:51:58.538211 19974 webserver.cc:460] Webserver started at http://127.12.15.66:42135/ using document root <none> and password file <none>
I20260806 03:51:58.538456 19974 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:58.538503 19974 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:58.538635 19974 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:58.539639 19974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "f0d597435e4b483fab8329f5a99bcfcd"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "43428508b2444300a8f8ee9e5b3771a0"
server_key_iv: "d00a5216e8ad3828cb96bf0a621870e8"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.539988 19974 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "f0d597435e4b483fab8329f5a99bcfcd"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "43428508b2444300a8f8ee9e5b3771a0"
server_key_iv: "d00a5216e8ad3828cb96bf0a621870e8"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.541397 19974 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:58.542092 20001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.542238 19974 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:58.542315 19974 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "f0d597435e4b483fab8329f5a99bcfcd"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "43428508b2444300a8f8ee9e5b3771a0"
server_key_iv: "d00a5216e8ad3828cb96bf0a621870e8"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.542382 19974 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:58.545701 19974 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:58.545940 19974 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:58.546056 19974 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:58.546279 19974 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:58.546561 19974 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:58.546592 19974 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.546612 19974 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:58.546634 19974 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.552062 19974 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:36759
I20260806 03:51:58.552121 20182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:36759 every 8 connection(s)
I20260806 03:51:58.552413 19974 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:51:58.555160 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.553114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:49949 (local address 127.12.15.126:36829)
0806 03:51:58.553177 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.553184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.553387 (+ 203us) server_negotiation.cc:407] Connection header received
0806 03:51:58.553831 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.553832 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.553836 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.553890 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.554976 (+ 1086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.554977 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.554977 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.554980 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.554995 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.554999 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.555041 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.555094 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.555105 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:51:58.555326 20190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.553086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36829 (local address 127.12.15.66:49949)
0806 03:51:58.553266 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.553305 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.553755 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.553946 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.553949 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.553957 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:58.554840 (+ 883us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:51:58.554841 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:58.554868 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:51:58.554898 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:58.555040 (+ 142us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:58.555042 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:58.555045 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.555098 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.555143 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":87,"threads_started":1}
I20260806 03:51:58.555781 20186 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36829
I20260806 03:51:58.555864 20186 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:58.556001 20186 heartbeater.cc:511] Master 127.12.15.126:36829 requested a full tablet report, sending...
I20260806 03:51:58.556324 19672 ts_manager.cc:194] Registered new tserver with Master: f0d597435e4b483fab8329f5a99bcfcd (127.12.15.66:36759)
I20260806 03:51:58.562268 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 19974
I20260806 03:51:58.562361 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:51:58.562532 12349 external_mini_cluster.cc:1428] Setting key 6968af22986e692a82d2c4b4711d5b8a
I20260806 03:51:58.563410 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36829
--builtin_ntp_servers=127.12.15.84:41843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260806 03:51:58.644057 20191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:58.644266 20191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:58.644299 20191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:58.645506 20191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:58.645552 20191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:58.645591 20191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:51:58.646958 20191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36829
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:58.647197 20191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:58.647491 20191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:58.652364 20199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:58.652421 20191 server_base.cc:1034] running on GCE node
W20260806 03:51:58.652354 20197 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:58.652369 20196 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:58.652745 20191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:58.652987 20191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:58.654153 20191 hybrid_clock.cc:648] HybridClock initialized: now 1785988318654131 us; error 33 us; skew 500 ppm
I20260806 03:51:58.655354 20191 webserver.cc:460] Webserver started at http://127.12.15.67:39817/ using document root <none> and password file <none>
I20260806 03:51:58.655601 20191 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:58.655648 20191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:58.655777 20191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:58.656813 20191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "c9339547334c4c40bfa25b24914ec54a"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "a6c1ef2182a69b9ece0755f0f2366025"
server_key_iv: "3eac5ed9e04e95db21573c21d18697d7"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.657137 20191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "c9339547334c4c40bfa25b24914ec54a"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "a6c1ef2182a69b9ece0755f0f2366025"
server_key_iv: "3eac5ed9e04e95db21573c21d18697d7"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.658329 20191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:58.658957 20213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.659111 20191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:58.659200 20191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "c9339547334c4c40bfa25b24914ec54a"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "a6c1ef2182a69b9ece0755f0f2366025"
server_key_iv: "3eac5ed9e04e95db21573c21d18697d7"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.659255 20191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:58.662465 20191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:58.662670 20191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:58.662761 20191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:58.662916 20191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:51:58.663153 20191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:51:58.663189 20191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.663208 20191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:51:58.663224 20191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.668524 20191 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:42663
I20260806 03:51:58.668596 20409 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:42663 every 8 connection(s)
I20260806 03:51:58.668857 20191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:51:58.671561 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.669512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:58023 (local address 127.12.15.126:36829)
0806 03:51:58.669586 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.669588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.669813 (+ 225us) server_negotiation.cc:407] Connection header received
0806 03:51:58.670284 (+ 471us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.670285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.670291 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.670342 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.671390 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.671390 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.671391 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.671393 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.671406 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.671409 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.671448 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.671495 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.671506 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:51:58.671727 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.669498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36829 (local address 127.12.15.67:58023)
0806 03:51:58.669702 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.669736 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.670201 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.670396 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.670400 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.670407 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:58.671269 (+ 862us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:51:58.671269 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:58.671296 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:51:58.671324 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:58.671448 (+ 124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:58.671450 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:58.671452 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.671497 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.671543 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":105,"threads_started":1}
I20260806 03:51:58.672173 20410 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36829
I20260806 03:51:58.672261 20410 heartbeater.cc:464] Registering TS with master...
I20260806 03:51:58.672405 20410 heartbeater.cc:511] Master 127.12.15.126:36829 requested a full tablet report, sending...
I20260806 03:51:58.672771 19672 ts_manager.cc:194] Registered new tserver with Master: c9339547334c4c40bfa25b24914ec54a (127.12.15.67:42663)
I20260806 03:51:58.678411 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 20191
I20260806 03:51:58.678494 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:51:58.678651 12349 external_mini_cluster.cc:1428] Setting key 8cebc50ba88cb1b4e42d7fdad81c4a0f
I20260806 03:51:58.679375 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:51:58.681497 19731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.680615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47556 (local address 127.12.15.126:36829)
0806 03:51:58.680665 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.680667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.680752 (+ 85us) server_negotiation.cc:407] Connection header received
0806 03:51:58.680795 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.680796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.680800 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.680842 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.681306 (+ 464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.681308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.681308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.681312 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.681326 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.681330 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.681369 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.681416 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.681428 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:51:58.682927 19672 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:51:58.684435 19672 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:51:58.687863 20421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.686370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:36759 (local address 127.0.0.1:39400)
0806 03:51:58.686523 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.686558 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.686610 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.686980 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.686980 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.686983 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:58.687459 (+ 476us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:51:58.687459 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:58.687475 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:51:58.687498 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:58.687711 (+ 213us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:58.687718 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:58.687721 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.687785 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.687799 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"spinlock_wait_cycles":384,"thread_start_us":94,"threads_started":1}
I20260806 03:51:58.688081 20422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.686369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:42663 (local address 127.0.0.1:47120)
0806 03:51:58.686523 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.686558 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.686610 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.686951 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.686957 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.686964 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:58.687720 (+ 756us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:51:58.687721 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:58.687728 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:51:58.687740 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:58.687984 (+ 244us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:58.687985 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:58.687986 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.688017 (+ 31us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.688029 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":2,"thread_start_us":51,"threads_started":1}
I20260806 03:51:58.688138 20428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.686663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39400 (local address 127.12.15.66:36759)
0806 03:51:58.686844 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.686849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.686857 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:51:58.686881 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.686883 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.686890 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.686962 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.687567 (+ 605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.687575 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.687576 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.687580 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.687615 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.687641 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.687696 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.688043 (+ 347us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.688059 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":89,"threads_started":1}
I20260806 03:51:58.688246 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.686662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35510 (local address 127.12.15.65:40345)
0806 03:51:58.686852 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.686857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.686867 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:51:58.686889 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.686891 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.686898 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.686961 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.687820 (+ 859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.687828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.687829 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.687832 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.687863 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.687878 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.687927 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.688138 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.688157 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":91,"threads_started":1}
I20260806 03:51:58.688369 20427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.686504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47120 (local address 127.12.15.67:42663)
0806 03:51:58.686810 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.686814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.686824 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:51:58.686844 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.686846 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.686855 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.687615 (+ 760us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.687845 (+ 230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.687848 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.687848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.687851 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.687879 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.687893 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.688261 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.688283 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.688297 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":75,"threads_started":1}
I20260806 03:51:58.688663 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.686394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40345 (local address 127.0.0.1:35510)
0806 03:51:58.686687 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.686695 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.686722 (+ 27us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.687209 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.687210 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.687212 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:58.687733 (+ 521us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:51:58.687734 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:58.687738 (+ 4us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:51:58.687746 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:58.688075 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:58.688076 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:58.688077 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.688587 (+ 510us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.688602 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":100,"threads_started":1}
I20260806 03:51:58.688789 20079 tablet_service.cc:1467] Processing CreateTablet for tablet 0239e575c80c4cf79614046d277d2f66 (DEFAULT_TABLE table=test-table [id=16106aa99d574c728827f0746254d42e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:58.688933 20301 tablet_service.cc:1467] Processing CreateTablet for tablet 0239e575c80c4cf79614046d277d2f66 (DEFAULT_TABLE table=test-table [id=16106aa99d574c728827f0746254d42e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:58.689198 20079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0239e575c80c4cf79614046d277d2f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:58.689255 20301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0239e575c80c4cf79614046d277d2f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:58.689332 19862 tablet_service.cc:1467] Processing CreateTablet for tablet 0239e575c80c4cf79614046d277d2f66 (DEFAULT_TABLE table=test-table [id=16106aa99d574c728827f0746254d42e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:51:58.689632 19862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0239e575c80c4cf79614046d277d2f66. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:58.691648 20431 tablet_bootstrap.cc:492] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: Bootstrap starting.
I20260806 03:51:58.692185 20431 tablet_bootstrap.cc:654] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:58.692349 20436 tablet_bootstrap.cc:492] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: Bootstrap starting.
I20260806 03:51:58.692477 20431 log.cc:824] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:58.692822 20438 tablet_bootstrap.cc:492] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89: Bootstrap starting.
I20260806 03:51:58.692950 20436 tablet_bootstrap.cc:654] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:58.693282 20436 log.cc:824] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:58.693455 20438 tablet_bootstrap.cc:654] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:58.693730 20438 log.cc:824] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:58.694161 20436 tablet_bootstrap.cc:492] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: No bootstrap required, opened a new log
I20260806 03:51:58.694231 20436 ts_tablet_manager.cc:1397] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:58.694413 20438 tablet_bootstrap.cc:492] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89: No bootstrap required, opened a new log
I20260806 03:51:58.694478 20438 ts_tablet_manager.cc:1397] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:51:58.694832 20431 tablet_bootstrap.cc:492] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: No bootstrap required, opened a new log
I20260806 03:51:58.694890 20431 ts_tablet_manager.cc:1397] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 03:51:58.696061 20436 raft_consensus.cc:348] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.696224 20436 raft_consensus.cc:374] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:58.696259 20436 raft_consensus.cc:729] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9339547334c4c40bfa25b24914ec54a, State: Initialized, Role: FOLLOWER
I20260806 03:51:58.696391 20436 consensus_queue.cc:260] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.696642 20436 ts_tablet_manager.cc:1428] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:51:58.696749 20410 heartbeater.cc:503] Master 127.12.15.126:36829 was elected leader, sending a full tablet report...
I20260806 03:51:58.696892 20438 raft_consensus.cc:348] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.697043 20438 raft_consensus.cc:374] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:58.697072 20438 raft_consensus.cc:729] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c72ccc3c7a8c4431a828efbd78f97a89, State: Initialized, Role: FOLLOWER
I20260806 03:51:58.697177 20438 consensus_queue.cc:260] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.697259 20431 raft_consensus.cc:348] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.697396 20431 raft_consensus.cc:374] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:58.697429 20431 raft_consensus.cc:729] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0d597435e4b483fab8329f5a99bcfcd, State: Initialized, Role: FOLLOWER
I20260806 03:51:58.697461 19971 heartbeater.cc:503] Master 127.12.15.126:36829 was elected leader, sending a full tablet report...
I20260806 03:51:58.697660 20431 consensus_queue.cc:260] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.697974 20431 ts_tablet_manager.cc:1428] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:51:58.697427 20438 ts_tablet_manager.cc:1428] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:51:58.699736 20186 heartbeater.cc:503] Master 127.12.15.126:36829 was elected leader, sending a full tablet report...
I20260806 03:51:58.699738 20453 raft_consensus.cc:482] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:51:58.699842 20453 raft_consensus.cc:504] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.700434 20453 leader_election.cc:288] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f0d597435e4b483fab8329f5a99bcfcd (127.12.15.66:36759), c72ccc3c7a8c4431a828efbd78f97a89 (127.12.15.65:40345)
I20260806 03:51:58.701916 20428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.700377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:36001 (local address 127.12.15.66:36759)
0806 03:51:58.701048 (+ 671us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.701049 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.701056 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:51:58.701073 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.701074 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.701077 (+ 3us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.701120 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.701705 (+ 585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.701707 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.701707 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.701711 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.701726 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.701730 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.701776 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.701843 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.701855 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649}
I20260806 03:51:58.701932 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.700284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:36759 (local address 127.12.15.67:36001)
0806 03:51:58.700865 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.700875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.700912 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.701164 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.701164 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.701167 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:58.701601 (+ 434us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:51:58.701601 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:58.701608 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:51:58.701620 (+ 12us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:58.701798 (+ 178us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:58.701799 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:58.701800 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.701842 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.701855 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564}
I20260806 03:51:58.702188 20429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.700670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:40399 (local address 127.12.15.65:40345)
0806 03:51:58.700708 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.700710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.701025 (+ 315us) server_negotiation.cc:407] Connection header received
0806 03:51:58.701048 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.701049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.701054 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.701109 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.701818 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.701819 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.701819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.701823 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.701840 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.701844 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.701899 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.702101 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.702114 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:51:58.702037 20413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.700654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40345 (local address 127.12.15.67:40399)
0806 03:51:58.700964 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.700970 (+ 6us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.700994 (+ 24us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.701130 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.701131 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.701134 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:51:58.701729 (+ 595us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:51:58.701730 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:51:58.701736 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:51:58.701744 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:51:58.701926 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:51:58.701927 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:51:58.701927 (+ 0us) client_negotiation.cc:769] Sending connection context
0806 03:51:58.701966 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 03:51:58.701979 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":100,"threads_started":1}
I20260806 03:51:58.702498 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0239e575c80c4cf79614046d277d2f66" candidate_uuid: "c9339547334c4c40bfa25b24914ec54a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" is_pre_election: true
I20260806 03:51:58.702497 20114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0239e575c80c4cf79614046d277d2f66" candidate_uuid: "c9339547334c4c40bfa25b24914ec54a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d597435e4b483fab8329f5a99bcfcd" is_pre_election: true
I20260806 03:51:58.702616 20114 raft_consensus.cc:2454] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9339547334c4c40bfa25b24914ec54a in term 0.
I20260806 03:51:58.702616 19902 raft_consensus.cc:2454] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c9339547334c4c40bfa25b24914ec54a in term 0.
I20260806 03:51:58.702841 20221 leader_election.cc:302] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c9339547334c4c40bfa25b24914ec54a, f0d597435e4b483fab8329f5a99bcfcd; no voters:
I20260806 03:51:58.703276 20446 raft_consensus.cc:2781] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:51:58.703364 20446 raft_consensus.cc:482] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:51:58.703397 20446 raft_consensus.cc:3037] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:58.704159 20446 raft_consensus.cc:504] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.704286 20446 leader_election.cc:288] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [CANDIDATE]: Term 1 election: Requested vote from peers f0d597435e4b483fab8329f5a99bcfcd (127.12.15.66:36759), c72ccc3c7a8c4431a828efbd78f97a89 (127.12.15.65:40345)
I20260806 03:51:58.704418 19902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0239e575c80c4cf79614046d277d2f66" candidate_uuid: "c9339547334c4c40bfa25b24914ec54a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c72ccc3c7a8c4431a828efbd78f97a89"
I20260806 03:51:58.704485 19902 raft_consensus.cc:3037] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:58.704613 20114 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0239e575c80c4cf79614046d277d2f66" candidate_uuid: "c9339547334c4c40bfa25b24914ec54a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f0d597435e4b483fab8329f5a99bcfcd"
I20260806 03:51:58.704694 20114 raft_consensus.cc:3037] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:58.705291 19902 raft_consensus.cc:2454] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9339547334c4c40bfa25b24914ec54a in term 1.
I20260806 03:51:58.705482 20114 raft_consensus.cc:2454] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c9339547334c4c40bfa25b24914ec54a in term 1.
I20260806 03:51:58.705468 20216 leader_election.cc:302] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c72ccc3c7a8c4431a828efbd78f97a89, c9339547334c4c40bfa25b24914ec54a; no voters:
I20260806 03:51:58.705690 20446 raft_consensus.cc:2781] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:58.705758 20446 raft_consensus.cc:686] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 1 LEADER]: Becoming Leader. State: Replica: c9339547334c4c40bfa25b24914ec54a, State: Running, Role: LEADER
I20260806 03:51:58.705827 20446 consensus_queue.cc:237] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } }
I20260806 03:51:58.706521 19670 catalog_manager.cc:5495] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a reported cstate change: term changed from 0 to 1, leader changed from <none> to c9339547334c4c40bfa25b24914ec54a (127.12.15.67). New cstate: current_term: 1 leader_uuid: "c9339547334c4c40bfa25b24914ec54a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9339547334c4c40bfa25b24914ec54a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42663 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 } health_report { overall_health: UNKNOWN } } }
I20260806 03:51:58.713100 20427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.712344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47130 (local address 127.12.15.67:42663)
0806 03:51:58.712385 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.712386 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.712393 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:51:58.712445 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.712445 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.712455 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.712495 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.712971 (+ 476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.712971 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.712972 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.712974 (+ 2us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.712982 (+ 8us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.712985 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.713017 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.713047 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.713056 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:51:58.714564 20114 raft_consensus.cc:1261] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 1 FOLLOWER]: Refusing update from remote peer c9339547334c4c40bfa25b24914ec54a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:58.714569 19902 raft_consensus.cc:1261] T 0239e575c80c4cf79614046d277d2f66 P c72ccc3c7a8c4431a828efbd78f97a89 [term 1 FOLLOWER]: Refusing update from remote peer c9339547334c4c40bfa25b24914ec54a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:51:58.714845 20453 consensus_queue.cc:1035] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f0d597435e4b483fab8329f5a99bcfcd" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36759 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:58.714962 20446 consensus_queue.cc:1035] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c72ccc3c7a8c4431a828efbd78f97a89" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40345 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:51:58.717252 20458 mvcc.cc:205] Tried to move back new op lower bound from 7315408153452597248 to 7315408153419644928. Current Snapshot: MvccSnapshot[applied={T|T < 7315408153452597248}]
I20260806 03:51:58.719122 19672 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:58.719214 19672 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:51:58.719489 20459 mvcc.cc:205] Tried to move back new op lower bound from 7315408153452597248 to 7315408153419644928. Current Snapshot: MvccSnapshot[applied={T|T < 7315408153452597248 or (T in {7315408153452597248})}]
I20260806 03:51:58.719883 19672 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ba345a38123244d4a02a1d1316f1ee6f: Sending DeleteTablet for 3 replicas of tablet 0239e575c80c4cf79614046d277d2f66
I20260806 03:51:58.720216 19862 tablet_service.cc:1514] Processing DeleteTablet for tablet 0239e575c80c4cf79614046d277d2f66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:58 UTC) from {username='slave'} at 127.0.0.1:35510
I20260806 03:51:58.720208 20301 tablet_service.cc:1514] Processing DeleteTablet for tablet 0239e575c80c4cf79614046d277d2f66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:58 UTC) from {username='slave'} at 127.0.0.1:47120
I20260806 03:51:58.720256 20079 tablet_service.cc:1514] Processing DeleteTablet for tablet 0239e575c80c4cf79614046d277d2f66 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:51:58 UTC) from {username='slave'} at 127.0.0.1:39400
I20260806 03:51:58.720470 20480 tablet_replica.cc:330] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: stopping tablet replica
I20260806 03:51:58.720599 20480 raft_consensus.cc:2229] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:51:58.720672 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 19765
I20260806 03:51:58.730665 20477 tablet_replica.cc:330] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: stopping tablet replica
I20260806 03:51:58.730777 20477 raft_consensus.cc:2229] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:51:58.730860 20477 raft_consensus.cc:2258] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:58.733729 20477 ts_tablet_manager.cc:1905] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:58.735705 20477 ts_tablet_manager.cc:1918] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:51:58.735770 20477 log.cc:1196] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/0239e575c80c4cf79614046d277d2f66
I20260806 03:51:58.736019 20477 ts_tablet_manager.cc:1939] T 0239e575c80c4cf79614046d277d2f66 P c9339547334c4c40bfa25b24914ec54a: Deleting consensus metadata
I20260806 03:51:58.736438 19643 catalog_manager.cc:4841] TS c9339547334c4c40bfa25b24914ec54a (127.12.15.67:42663): tablet 0239e575c80c4cf79614046d277d2f66 (table test-table [id=16106aa99d574c728827f0746254d42e]) successfully deleted
I20260806 03:51:58.744293 20480 raft_consensus.cc:2258] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:51:58.745033 20480 ts_tablet_manager.cc:1905] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:51:58.746846 20480 ts_tablet_manager.cc:1918] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:51:58.746948 20480 log.cc:1196] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/0239e575c80c4cf79614046d277d2f66
I20260806 03:51:58.747241 20480 ts_tablet_manager.cc:1939] T 0239e575c80c4cf79614046d277d2f66 P f0d597435e4b483fab8329f5a99bcfcd: Deleting consensus metadata
I20260806 03:51:58.747859 19648 catalog_manager.cc:4841] TS f0d597435e4b483fab8329f5a99bcfcd (127.12.15.66:36759): tablet 0239e575c80c4cf79614046d277d2f66 (table test-table [id=16106aa99d574c728827f0746254d42e]) successfully deleted
W20260806 03:51:58.748871 19645 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:40345 (error 108)
I20260806 03:51:58.749127 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 19974
I20260806 03:51:58.758981 20425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.758819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40345 (local address 127.0.0.1:35518)
0806 03:51:58.758883 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.758940 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:40345: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260806 03:51:58.759120 19645 catalog_manager.cc:4584] TS c72ccc3c7a8c4431a828efbd78f97a89 (127.12.15.65:40345): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0239e575c80c4cf79614046d277d2f66: Network error: Client connection negotiation failed: client connection to 127.12.15.65:40345: connect: Connection refused (error 111)
I20260806 03:51:58.760680 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 20191
I20260806 03:51:58.772300 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 19621
2026-08-06T03:51:58Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (632 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260806 03:51:58.795097 12349 mini_oidc.cc:152] Starting JWKS server
I20260806 03:51:58.796288 12349 webserver.cc:460] Webserver started at https://127.0.0.1:39203/ using document root <none> and password file <none>
I20260806 03:51:58.796373 12349 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 03:51:58.796530 12349 webserver.cc:460] Webserver started at http://127.0.0.1:42499/ using document root <none> and password file <none>
2026-08-06T03:51:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:58Z Disabled control of system clock
I20260806 03:51:58.804703 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35933
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:44119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:35933
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39203/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/testchainca.cert with env {}
W20260806 03:51:58.883428 20493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:58.883628 20493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:58.883649 20493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:58.884737 20493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:58.884769 20493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:58.884781 20493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:58.884791 20493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:58.884806 20493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 03:51:58.884817 20493 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 03:51:58.884831 20493 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39203/jwks.json
I20260806 03:51:58.886339 20493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:35933
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35933
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39203/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:58.886514 20493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:58.886678 20493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:58.891146 20500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:58.891197 20493 server_base.cc:1034] running on GCE node
W20260806 03:51:58.891155 20502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:58.891166 20499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:58.891605 20493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:58.891858 20493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:58.893019 20493 hybrid_clock.cc:648] HybridClock initialized: now 1785988318893002 us; error 34 us; skew 500 ppm
I20260806 03:51:58.894199 20493 webserver.cc:460] Webserver started at http://127.12.15.126:37507/ using document root <none> and password file <none>
I20260806 03:51:58.894402 20493 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:58.894441 20493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:58.894531 20493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:58.895463 20493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "9c6ff3d8930f4e96be3b4cc5502b8310"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "8a0ed539ae1e9139f90d8f4603f519c3"
server_key_iv: "4505a3823a6616e47f8a3bc02a5ebffe"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.895756 20493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "9c6ff3d8930f4e96be3b4cc5502b8310"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "8a0ed539ae1e9139f90d8f4603f519c3"
server_key_iv: "4505a3823a6616e47f8a3bc02a5ebffe"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.896914 20493 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:51:58.897516 20515 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:58.897673 20493 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:58.897755 20493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "9c6ff3d8930f4e96be3b4cc5502b8310"
format_stamp: "Formatted at 2026-08-06 03:51:58 on dist-test-slave-bqcl"
server_key: "8a0ed539ae1e9139f90d8f4603f519c3"
server_key_iv: "4505a3823a6616e47f8a3bc02a5ebffe"
server_key_version: "encryptionkey@0"
I20260806 03:51:58.897809 20493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:58.919585 20493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:58.919849 20493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:58.919947 20493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:58.923676 20493 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:35933
I20260806 03:51:58.923715 20596 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:35933 every 8 connection(s)
I20260806 03:51:58.924008 20493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:58.924964 20597 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:58.927191 20597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310: Bootstrap starting.
I20260806 03:51:58.927717 20597 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:58.927949 20597 log.cc:824] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:58.928524 20597 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310: No bootstrap required, opened a new log
I20260806 03:51:58.928752 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 20493
I20260806 03:51:58.928848 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:58.929009 12349 external_mini_cluster.cc:1428] Setting key a024ff138434bb13d327a56c29df33e9
I20260806 03:51:58.930680 20597 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6ff3d8930f4e96be3b4cc5502b8310" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35933 } }
I20260806 03:51:58.930822 20597 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:58.930856 20597 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c6ff3d8930f4e96be3b4cc5502b8310, State: Initialized, Role: FOLLOWER
I20260806 03:51:58.931005 20597 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6ff3d8930f4e96be3b4cc5502b8310" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35933 } }
I20260806 03:51:58.931093 20597 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:58.931125 20597 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:58.931174 20597 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:58.931813 20597 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6ff3d8930f4e96be3b4cc5502b8310" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35933 } }
I20260806 03:51:58.931917 20597 leader_election.cc:302] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9c6ff3d8930f4e96be3b4cc5502b8310; no voters:
I20260806 03:51:58.932071 20597 leader_election.cc:288] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:58.932150 20604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:58.932287 20604 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [term 1 LEADER]: Becoming Leader. State: Replica: 9c6ff3d8930f4e96be3b4cc5502b8310, State: Running, Role: LEADER
I20260806 03:51:58.932304 20597 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:58.932472 20604 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6ff3d8930f4e96be3b4cc5502b8310" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35933 } }
I20260806 03:51:58.934417 20606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9c6ff3d8930f4e96be3b4cc5502b8310. Latest consensus state: current_term: 1 leader_uuid: "9c6ff3d8930f4e96be3b4cc5502b8310" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6ff3d8930f4e96be3b4cc5502b8310" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35933 } } }
I20260806 03:51:58.934494 20606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [sys.catalog]: This master's current role is: LEADER
W20260806 03:51:58.934012 20626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:51:58.934650 20626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:51:58.934767 20605 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9c6ff3d8930f4e96be3b4cc5502b8310" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9c6ff3d8930f4e96be3b4cc5502b8310" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35933 } } }
I20260806 03:51:58.934845 20605 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:58.934785 20629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:58.934998 20629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:58.936029 20603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:58.929503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.12.15.126:35933)
0806 03:51:58.929681 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:58.929688 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:58.929713 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:51:58.930657 (+ 944us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:58.930665 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:58.930729 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:58.930858 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:58.931120 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.932533 (+ 1413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.933699 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.934284 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.934401 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:58.935311 (+ 910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:58.935315 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:58.935315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:58.935529 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:58.935575 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:58.935591 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:58.935655 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:58.935722 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 03:51:58.935800 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":64,"threads_started":1}
I20260806 03:51:58.936553 20629 catalog_manager.cc:1347] Generated new cluster ID: bc2ae94e65d44677adb95058150b3f1b
I20260806 03:51:58.936647 20629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:58.936941 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:58.946187 20629 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:58.946681 20629 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:58.951643 20629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9c6ff3d8930f4e96be3b4cc5502b8310: Generated new TSK 0
I20260806 03:51:58.951792 20629 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 03:51:58.976496 20639 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260806 03:51:58.976653 20639 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260806 03:51:58.977236 20638 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:58.959219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35933 (local address 127.0.0.1:41818)
0806 03:51:58.959389 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:58.959399 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:58.959461 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:58.959629 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:58.959630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:58.959632 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 03:51:58.959690 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:58.959692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.960285 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.960286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:58.960858 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:58.960860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:58.961185 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:58.961185 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:58.961252 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:58.961271 (+ 19us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 03:51:58.976713 (+ 15442us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0806 03:51:58.976831 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.15.126:35933: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":81,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.12.15.126:35933: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39203/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260806 03:51:58.977669 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 20493
2026-08-06T03:51:59Z chronyd exiting
I20260806 03:51:59.021235 12349 test_util.cc:175] -----------------------------------------------
I20260806 03:51:59.021341 12349 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1785988295328500-12349-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260806 03:51:59.022433 12349 mini_oidc.cc:152] Starting JWKS server
I20260806 03:51:59.023074 12349 webserver.cc:460] Webserver started at https://127.0.0.1:39513/ using document root <none> and password file <none>
I20260806 03:51:59.023113 12349 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 03:51:59.023221 12349 webserver.cc:460] Webserver started at http://127.0.0.1:37081/ using document root <none> and password file <none>
2026-08-06T03:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:59Z Disabled control of system clock
I20260806 03:51:59.031541 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35033
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:40955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:35033
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39513/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/testchainca.cert with env {}
W20260806 03:51:59.110409 20647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:59.110610 20647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:59.110639 20647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:59.111845 20647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:59.111886 20647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:59.111901 20647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:59.111917 20647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:59.111931 20647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 03:51:59.111948 20647 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 03:51:59.111970 20647 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39513/jwks.json
I20260806 03:51:59.113425 20647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:35033
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39513/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:59.113662 20647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:59.113868 20647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:59.118353 20659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:59.118362 20653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:59.118366 20656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:59.118593 20647 server_base.cc:1034] running on GCE node
I20260806 03:51:59.118840 20647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:59.119089 20647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:59.120249 20647 hybrid_clock.cc:648] HybridClock initialized: now 1785988319120231 us; error 31 us; skew 500 ppm
I20260806 03:51:59.121415 20647 webserver.cc:460] Webserver started at http://127.12.15.126:37773/ using document root <none> and password file <none>
I20260806 03:51:59.121651 20647 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:59.121698 20647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:59.121819 20647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:59.122860 20647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "23af51bc07b44d8194f3ea06897987f1"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "9744ce792ec171d97e6a87775b270706"
server_key_iv: "dc02076b1ddb130f2a24bc11591cec8e"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.123247 20647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "23af51bc07b44d8194f3ea06897987f1"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "9744ce792ec171d97e6a87775b270706"
server_key_iv: "dc02076b1ddb130f2a24bc11591cec8e"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.124588 20647 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:51:59.125241 20666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:59.125382 20647 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:59.125465 20647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "23af51bc07b44d8194f3ea06897987f1"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "9744ce792ec171d97e6a87775b270706"
server_key_iv: "dc02076b1ddb130f2a24bc11591cec8e"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.125535 20647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:59.140848 20647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:59.141145 20647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:59.141271 20647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:59.145128 20647 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:35033
I20260806 03:51:59.145161 20754 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:35033 every 8 connection(s)
I20260806 03:51:59.145505 20647 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:59.146317 20757 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:59.149329 20757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1: Bootstrap starting.
I20260806 03:51:59.149866 20757 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:59.150099 20757 log.cc:824] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:59.150816 20757 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1: No bootstrap required, opened a new log
I20260806 03:51:59.152338 20757 raft_consensus.cc:348] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23af51bc07b44d8194f3ea06897987f1" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35033 } }
I20260806 03:51:59.152475 20757 raft_consensus.cc:374] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:59.152496 20757 raft_consensus.cc:729] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23af51bc07b44d8194f3ea06897987f1, State: Initialized, Role: FOLLOWER
I20260806 03:51:59.152613 20757 consensus_queue.cc:260] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23af51bc07b44d8194f3ea06897987f1" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35033 } }
I20260806 03:51:59.152693 20757 raft_consensus.cc:388] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:59.152720 20757 raft_consensus.cc:482] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:59.152753 20757 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:59.153230 20757 raft_consensus.cc:504] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23af51bc07b44d8194f3ea06897987f1" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35033 } }
I20260806 03:51:59.153316 20757 leader_election.cc:302] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 23af51bc07b44d8194f3ea06897987f1; no voters:
I20260806 03:51:59.153443 20757 leader_election.cc:288] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:59.153487 20762 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:59.153669 20757 sys_catalog.cc:564] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:59.153678 20762 raft_consensus.cc:686] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [term 1 LEADER]: Becoming Leader. State: Replica: 23af51bc07b44d8194f3ea06897987f1, State: Running, Role: LEADER
I20260806 03:51:59.153803 20762 consensus_queue.cc:237] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23af51bc07b44d8194f3ea06897987f1" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35033 } }
I20260806 03:51:59.154222 20768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 23af51bc07b44d8194f3ea06897987f1. Latest consensus state: current_term: 1 leader_uuid: "23af51bc07b44d8194f3ea06897987f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23af51bc07b44d8194f3ea06897987f1" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35033 } } }
I20260806 03:51:59.154318 20768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:59.154402 20764 sys_catalog.cc:455] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "23af51bc07b44d8194f3ea06897987f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "23af51bc07b44d8194f3ea06897987f1" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35033 } } }
I20260806 03:51:59.154485 20764 sys_catalog.cc:458] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:59.154608 20778 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:59.155121 20778 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:59.155645 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 20647
I20260806 03:51:59.155743 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:59.155905 12349 external_mini_cluster.cc:1428] Setting key bd6ee45304eb5bf35440ad5d710d2d2c
I20260806 03:51:59.156900 20778 catalog_manager.cc:1347] Generated new cluster ID: 7b05a5c0cb6741f7aa1a6848193ce142
I20260806 03:51:59.156953 20778 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:59.162077 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:59.156339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53894 (local address 127.12.15.126:35033)
0806 03:51:59.156550 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:59.156558 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:59.156812 (+ 254us) server_negotiation.cc:407] Connection header received
0806 03:51:59.156929 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:59.156936 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:59.157007 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:59.157147 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:59.157365 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.158805 (+ 1440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.159853 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.160456 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.160591 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.161242 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:59.161247 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:59.161247 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:59.161457 (+ 210us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:59.161501 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:59.161515 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:59.161599 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:59.161660 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 03:51:59.161743 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260806 03:51:59.162986 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:59.171801 20778 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:59.172271 20778 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:59.195288 20778 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 23af51bc07b44d8194f3ea06897987f1: Generated new TSK 0
I20260806 03:51:59.195528 20778 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260806 03:51:59.216907 20803 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260806 03:51:59.216979 20803 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260806 03:51:59.217337 20795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:59.202059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53908 (local address 127.12.15.126:35033)
0806 03:51:59.202102 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:59.202104 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:59.202181 (+ 77us) server_negotiation.cc:407] Connection header received
0806 03:51:59.202267 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:59.202267 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:59.202278 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:59.202335 (+ 57us) server_negotiation.cc:226] Negotiated authn=JWT
0806 03:51:59.202467 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.202963 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.204075 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.204262 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.204302 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.204472 (+ 170us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 03:51:59.217112 (+ 12640us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 03:51:59.217263 (+ 151us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53908: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 03:51:59.217389 20802 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:59.201955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:35033 (local address 127.0.0.1:53908)
0806 03:51:59.202097 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:59.202108 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:59.202199 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:59.202344 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:59.202345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:59.202347 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0806 03:51:59.202410 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:59.202413 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.203042 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.203043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:59.203999 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:59.204003 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.204325 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.204326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:59.204386 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.204392 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 03:51:59.217216 (+ 12824us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0806 03:51:59.217312 (+ 96us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.15.126:35033: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
W20260806 03:51:59.217401 20795 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53908: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39513/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260806 03:51:59.217712 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 20647
2026-08-06T03:51:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (241 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260806 03:51:59.263607 12349 mini_oidc.cc:152] Starting JWKS server
I20260806 03:51:59.264262 12349 webserver.cc:460] Webserver started at https://127.0.0.1:35127/ using document root <none> and password file <none>
I20260806 03:51:59.264304 12349 mini_oidc.cc:189] Starting OIDC Discovery server
I20260806 03:51:59.264421 12349 webserver.cc:460] Webserver started at http://127.0.0.1:37115/ using document root <none> and password file <none>
2026-08-06T03:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:59Z Disabled control of system clock
I20260806 03:51:59.273028 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37121
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:38579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:37121
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35127/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 03:51:59.352794 20816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:59.353005 20816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:59.353035 20816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:59.354192 20816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:59.354230 20816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:59.354247 20816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:59.354264 20816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:59.354279 20816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 03:51:59.354296 20816 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 03:51:59.354319 20816 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35127/jwks.json
I20260806 03:51:59.355652 20816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:37121
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37121
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35127/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:59.355885 20816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:59.356153 20816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:59.360726 20823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:59.360730 20824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:59.360807 20816 server_base.cc:1034] running on GCE node
W20260806 03:51:59.360738 20827 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:59.361181 20816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:59.361455 20816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:59.362622 20816 hybrid_clock.cc:648] HybridClock initialized: now 1785988319362601 us; error 31 us; skew 500 ppm
I20260806 03:51:59.363790 20816 webserver.cc:460] Webserver started at http://127.12.15.126:37185/ using document root <none> and password file <none>
I20260806 03:51:59.363991 20816 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:59.364039 20816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:59.364164 20816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:59.365253 20816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "fa5ff644b42d4284a99334fae94da4d2"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "f00e6cb5477392152bc346745109fff4"
server_key_iv: "a639eec06f64e2fccd07138e8d5e8a0c"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.365626 20816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "fa5ff644b42d4284a99334fae94da4d2"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "f00e6cb5477392152bc346745109fff4"
server_key_iv: "a639eec06f64e2fccd07138e8d5e8a0c"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.366914 20816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:51:59.367542 20836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:59.367689 20816 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:51:59.367770 20816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "fa5ff644b42d4284a99334fae94da4d2"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "f00e6cb5477392152bc346745109fff4"
server_key_iv: "a639eec06f64e2fccd07138e8d5e8a0c"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.367828 20816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:59.388294 20816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:59.388607 20816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:59.388737 20816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:59.392446 20816 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:37121
I20260806 03:51:59.392504 20920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:37121 every 8 connection(s)
I20260806 03:51:59.392814 20816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:59.393477 20924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:59.396220 20924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2: Bootstrap starting.
I20260806 03:51:59.396858 20924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:59.397127 20924 log.cc:824] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:59.397174 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 20816
I20260806 03:51:59.397264 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:59.397420 12349 external_mini_cluster.cc:1428] Setting key da24469f6d59b83f01e96c5e7b23d5de
I20260806 03:51:59.397815 20924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2: No bootstrap required, opened a new log
I20260806 03:51:59.400061 20924 raft_consensus.cc:348] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5ff644b42d4284a99334fae94da4d2" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37121 } }
I20260806 03:51:59.400197 20924 raft_consensus.cc:374] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:59.400218 20924 raft_consensus.cc:729] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa5ff644b42d4284a99334fae94da4d2, State: Initialized, Role: FOLLOWER
I20260806 03:51:59.400332 20924 consensus_queue.cc:260] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5ff644b42d4284a99334fae94da4d2" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37121 } }
I20260806 03:51:59.400398 20924 raft_consensus.cc:388] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:59.400427 20924 raft_consensus.cc:482] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:59.400475 20924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:59.400988 20924 raft_consensus.cc:504] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5ff644b42d4284a99334fae94da4d2" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37121 } }
I20260806 03:51:59.401087 20924 leader_election.cc:302] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fa5ff644b42d4284a99334fae94da4d2; no voters:
I20260806 03:51:59.401242 20924 leader_election.cc:288] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:59.401274 20939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:59.401413 20939 raft_consensus.cc:686] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [term 1 LEADER]: Becoming Leader. State: Replica: fa5ff644b42d4284a99334fae94da4d2, State: Running, Role: LEADER
I20260806 03:51:59.401484 20924 sys_catalog.cc:564] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:59.401499 20939 consensus_queue.cc:237] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5ff644b42d4284a99334fae94da4d2" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37121 } }
I20260806 03:51:59.401960 20940 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fa5ff644b42d4284a99334fae94da4d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5ff644b42d4284a99334fae94da4d2" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37121 } } }
I20260806 03:51:59.402405 20940 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:59.402868 20953 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:59.403183 20941 sys_catalog.cc:455] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fa5ff644b42d4284a99334fae94da4d2. Latest consensus state: current_term: 1 leader_uuid: "fa5ff644b42d4284a99334fae94da4d2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fa5ff644b42d4284a99334fae94da4d2" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37121 } } }
I20260806 03:51:59.403250 20941 sys_catalog.cc:458] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:59.403354 20953 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:59.403419 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:59.397900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.12.15.126:37121)
0806 03:51:59.398101 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:59.398108 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:59.398161 (+ 53us) server_negotiation.cc:407] Connection header received
0806 03:51:59.399018 (+ 857us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:59.399025 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:59.399084 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:59.399210 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:59.399417 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.400323 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.401092 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.401694 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.401844 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.402536 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:59.402540 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:59.402541 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:59.402755 (+ 214us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:59.402802 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:59.402818 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:59.402878 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:59.403129 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 03:51:59.403208 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":80,"threads_started":1}
I20260806 03:51:59.404426 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:59.404824 20953 catalog_manager.cc:1347] Generated new cluster ID: 7b7ae24a42c24ac4a6aa274099485cf2
I20260806 03:51:59.404879 20953 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:59.415742 20953 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:59.416411 20953 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:59.419514 20953 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fa5ff644b42d4284a99334fae94da4d2: Generated new TSK 0
I20260806 03:51:59.419682 20953 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:59.431908 20938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:59.428386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35152 (local address 127.12.15.126:37121)
0806 03:51:59.428430 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:59.428432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:59.428514 (+ 82us) server_negotiation.cc:407] Connection header received
0806 03:51:59.428588 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:59.428589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:59.428601 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:59.428667 (+ 66us) server_negotiation.cc:226] Negotiated authn=JWT
0806 03:51:59.428756 (+ 89us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.429511 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.430315 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.430638 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.430706 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.430801 (+ 95us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 03:51:59.431716 (+ 915us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35127/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 03:51:59.431842 (+ 126us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:35152: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35127/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 03:51:59.431964 20938 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:35152: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35127/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260806 03:51:59.433756 20976 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:59.428298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37121 (local address 127.0.0.1:35152)
0806 03:51:59.428450 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:59.428460 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:59.428512 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:59.428669 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:59.428670 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:59.428671 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 03:51:59.428717 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:59.428719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.429628 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.429628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:59.430250 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:59.430252 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.430702 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.430702 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:59.430735 (+ 33us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.430740 (+ 5us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 03:51:59.431807 (+ 1067us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35127/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0806 03:51:59.433657 (+ 1850us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.15.126:37121: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35127/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":121,"mutex_wait_us":1674,"thread_start_us":86,"threads_started":1}
I20260806 03:51:59.434067 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 20816
E20260806 03:51:59.451325 20978 webserver.cc:533] Webserver: sslize failed:
E20260806 03:51:59.451401 20978 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-06T03:51:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (216 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-06T03:51:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:51:59Z Disabled control of system clock
I20260806 03:51:59.489127 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:45785
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:33233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:45785
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/testchainca.cert with env {}
W20260806 03:51:59.569510 20983 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:51:59.569725 20983 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:51:59.569756 20983 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:51:59.570930 20983 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:51:59.570971 20983 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:51:59.570988 20983 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:51:59.571004 20983 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:51:59.571020 20983 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260806 03:51:59.571040 20983 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260806 03:51:59.571056 20983 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260806 03:51:59.572535 20983 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:45785
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:45785
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20983
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:51:59.572781 20983 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:51:59.572969 20983 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:51:59.577526 20996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:51:59.577528 20989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:59.577652 20983 server_base.cc:1034] running on GCE node
W20260806 03:51:59.577561 20991 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:51:59.577993 20983 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:51:59.578269 20983 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:51:59.579404 20983 hybrid_clock.cc:648] HybridClock initialized: now 1785988319579392 us; error 29 us; skew 500 ppm
I20260806 03:51:59.580549 20983 webserver.cc:460] Webserver started at http://127.12.15.126:32899/ using document root <none> and password file <none>
I20260806 03:51:59.580758 20983 fs_manager.cc:359] Metadata directory not provided
I20260806 03:51:59.580806 20983 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:51:59.580926 20983 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:51:59.581893 20983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "c16ee8771c354aa0b78f4c6bfe32c957"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "ed93dee7b979fc6fa3cd1a1d5a557a06"
server_key_iv: "99a6ef4704c2b67899e017246c681fb6"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.582298 20983 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "c16ee8771c354aa0b78f4c6bfe32c957"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "ed93dee7b979fc6fa3cd1a1d5a557a06"
server_key_iv: "99a6ef4704c2b67899e017246c681fb6"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.583503 20983 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:51:59.584123 21003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:51:59.584268 20983 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:51:59.584348 20983 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "c16ee8771c354aa0b78f4c6bfe32c957"
format_stamp: "Formatted at 2026-08-06 03:51:59 on dist-test-slave-bqcl"
server_key: "ed93dee7b979fc6fa3cd1a1d5a557a06"
server_key_iv: "99a6ef4704c2b67899e017246c681fb6"
server_key_version: "encryptionkey@0"
I20260806 03:51:59.584405 20983 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:51:59.620111 20983 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:51:59.620451 20983 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:51:59.620599 20983 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:51:59.624380 20983 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:45785
I20260806 03:51:59.624439 21083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:45785 every 8 connection(s)
I20260806 03:51:59.624768 20983 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:51:59.625779 21084 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:51:59.628218 21084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957: Bootstrap starting.
I20260806 03:51:59.628804 21084 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957: Neither blocks nor log segments found. Creating new log.
I20260806 03:51:59.629099 21084 log.cc:824] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957: Log is configured to *not* fsync() on all Append() calls
I20260806 03:51:59.629870 21084 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957: No bootstrap required, opened a new log
I20260806 03:51:59.631561 21084 raft_consensus.cc:348] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ee8771c354aa0b78f4c6bfe32c957" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45785 } }
I20260806 03:51:59.631708 21084 raft_consensus.cc:374] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:51:59.631747 21084 raft_consensus.cc:729] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c16ee8771c354aa0b78f4c6bfe32c957, State: Initialized, Role: FOLLOWER
I20260806 03:51:59.631894 21084 consensus_queue.cc:260] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ee8771c354aa0b78f4c6bfe32c957" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45785 } }
I20260806 03:51:59.631959 21084 raft_consensus.cc:388] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:51:59.632002 21084 raft_consensus.cc:482] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:51:59.632054 21084 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:51:59.632587 21084 raft_consensus.cc:504] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ee8771c354aa0b78f4c6bfe32c957" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45785 } }
I20260806 03:51:59.632689 21084 leader_election.cc:302] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c16ee8771c354aa0b78f4c6bfe32c957; no voters:
I20260806 03:51:59.632860 21084 leader_election.cc:288] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:51:59.632895 21089 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:51:59.633029 21089 raft_consensus.cc:686] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [term 1 LEADER]: Becoming Leader. State: Replica: c16ee8771c354aa0b78f4c6bfe32c957, State: Running, Role: LEADER
I20260806 03:51:59.633116 21084 sys_catalog.cc:564] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:51:59.633137 21089 consensus_queue.cc:237] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ee8771c354aa0b78f4c6bfe32c957" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45785 } }
I20260806 03:51:59.633368 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 20983
I20260806 03:51:59.633463 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:51:59.633570 21091 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c16ee8771c354aa0b78f4c6bfe32c957. Latest consensus state: current_term: 1 leader_uuid: "c16ee8771c354aa0b78f4c6bfe32c957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ee8771c354aa0b78f4c6bfe32c957" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45785 } } }
I20260806 03:51:59.633636 12349 external_mini_cluster.cc:1428] Setting key c7b9f4cd9353d64589e73037707f502c
I20260806 03:51:59.633632 21091 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:59.633652 21090 sys_catalog.cc:455] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c16ee8771c354aa0b78f4c6bfe32c957" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c16ee8771c354aa0b78f4c6bfe32c957" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45785 } } }
I20260806 03:51:59.633737 21090 sys_catalog.cc:458] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957 [sys.catalog]: This master's current role is: LEADER
I20260806 03:51:59.633850 21100 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:51:59.634378 21100 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:51:59.636029 21100 catalog_manager.cc:1347] Generated new cluster ID: b5ace56c08b445baaf79a49f22228ac9
I20260806 03:51:59.636082 21100 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:51:59.639412 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:59.634262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33722 (local address 127.12.15.126:45785)
0806 03:51:59.635003 (+ 741us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:59.635012 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:59.635048 (+ 36us) server_negotiation.cc:407] Connection header received
0806 03:51:59.635165 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:59.635173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:59.635240 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:59.635398 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:51:59.635704 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.636707 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.637524 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.637952 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.638078 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.638771 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:51:59.638775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:51:59.638775 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:51:59.638955 (+ 180us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:51:59.639000 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:51:59.639012 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:51:59.639074 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:51:59.639161 (+ 87us) server_negotiation.cc:299] Negotiation successful
0806 03:51:59.639240 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":654,"spinlock_wait_cycles":27008,"thread_start_us":97,"threads_started":1}
I20260806 03:51:59.640298 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:51:59.647979 21100 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:51:59.648463 21100 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:51:59.654839 21100 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c16ee8771c354aa0b78f4c6bfe32c957: Generated new TSK 0
I20260806 03:51:59.655030 21100 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:51:59.697836 21111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:51:59.664678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33736 (local address 127.12.15.126:45785)
0806 03:51:59.664730 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:51:59.664732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:51:59.664818 (+ 86us) server_negotiation.cc:407] Connection header received
0806 03:51:59.664858 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:51:59.664859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:51:59.664872 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:51:59.664940 (+ 68us) server_negotiation.cc:226] Negotiated authn=JWT
0806 03:51:59.665042 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.665804 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.666628 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.666930 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.666998 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.667225 (+ 227us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0806 03:51:59.697586 (+ 30361us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 03:51:59.697760 (+ 174us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33736: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":31}
W20260806 03:51:59.697906 21111 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33736: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260806 03:51:59.697924 21129 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:51:59.664611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:45785 (local address 127.0.0.1:33736)
0806 03:51:59.664745 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 03:51:59.664755 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:51:59.664812 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:51:59.664948 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:51:59.664949 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:51:59.664950 (+ 1us) client_negotiation.cc:189] Negotiated authn=JWT
0806 03:51:59.664996 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:59.664997 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.665889 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.665890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:59.666553 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:51:59.666557 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:51:59.667055 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:51:59.667056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:51:59.667110 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:51:59.667116 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0806 03:51:59.697721 (+ 30605us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0806 03:51:59.697839 (+ 118us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.12.15.126:45785: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":87,"threads_started":1}
I20260806 03:51:59.698323 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 20983
2026-08-06T03:51:59Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (242 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (88 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (88 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:51:59 dist-test-slave-bqcl krb5kdc[21150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:51:59 dist-test-slave-bqcl krb5kdc[21150](info): set up 2 sockets
Aug 06 03:51:59 dist-test-slave-bqcl krb5kdc[21150](info): commencing operation
krb5kdc: starting...
W20260806 03:52:01.001369 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:52:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:52:01.121021 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38891
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:38891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:01.202266 21175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:01.202487 21175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:01.202509 21175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:01.203709 21175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:01.203758 21175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:01.203771 21175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:01.203794 21175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:01.203816 21175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:01.205317 21175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:38891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:01.205566 21175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:01.205783 21175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:52:01.210767 21175 server_base.cc:1034] running on GCE node
W20260806 03:52:01.210791 21183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:01.210713 21182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:01.210721 21185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:01.211126 21175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:01.211426 21175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:01.212585 21175 hybrid_clock.cc:648] HybridClock initialized: now 1785988321212567 us; error 31 us; skew 500 ppm
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.215050 21175 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:52:01.215512 21175 webserver.cc:460] Webserver started at http://127.12.15.126:34019/ using document root <none> and password file <none>
I20260806 03:52:01.215721 21175 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:01.215771 21175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:01.215899 21175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:01.216939 21175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "f25341c9c79c45738bb5ccb2830c6022"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "1ac79e4076df905f0b5a86a105eba4d3"
server_key_iv: "51f37bd63abd38315c5c3dae717be9de"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.217278 21175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "f25341c9c79c45738bb5ccb2830c6022"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "1ac79e4076df905f0b5a86a105eba4d3"
server_key_iv: "51f37bd63abd38315c5c3dae717be9de"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.218640 21175 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:52:01.219321 21198 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.219467 21175 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:01.219554 21175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "f25341c9c79c45738bb5ccb2830c6022"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "1ac79e4076df905f0b5a86a105eba4d3"
server_key_iv: "51f37bd63abd38315c5c3dae717be9de"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.219616 21175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:01.234210 21175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:01.235858 21175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:01.235993 21175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:01.239753 21175 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:38891
I20260806 03:52:01.239794 21273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:38891 every 8 connection(s)
I20260806 03:52:01.240110 21175 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:52:01.240934 21274 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:01.243250 21274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022: Bootstrap starting.
I20260806 03:52:01.243824 21274 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:01.244083 21274 log.cc:824] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:01.244717 21274 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022: No bootstrap required, opened a new log
I20260806 03:52:01.245972 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 21175
I20260806 03:52:01.246054 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:52:01.246230 12349 external_mini_cluster.cc:1428] Setting key 30edb46a5cf5ba752170ac8b2fc18ef9
I20260806 03:52:01.246464 21274 raft_consensus.cc:348] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25341c9c79c45738bb5ccb2830c6022" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38891 } }
I20260806 03:52:01.246645 21274 raft_consensus.cc:374] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:01.246685 21274 raft_consensus.cc:729] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f25341c9c79c45738bb5ccb2830c6022, State: Initialized, Role: FOLLOWER
I20260806 03:52:01.246838 21274 consensus_queue.cc:260] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25341c9c79c45738bb5ccb2830c6022" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38891 } }
I20260806 03:52:01.247064 21274 raft_consensus.cc:388] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:52:01.247119 21274 raft_consensus.cc:482] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:52:01.247164 21274 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:01.247853 21274 raft_consensus.cc:504] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25341c9c79c45738bb5ccb2830c6022" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38891 } }
I20260806 03:52:01.247953 21274 leader_election.cc:302] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f25341c9c79c45738bb5ccb2830c6022; no voters:
I20260806 03:52:01.248113 21274 leader_election.cc:288] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:52:01.248165 21280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:01.248359 21274 sys_catalog.cc:564] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:01.248366 21280 raft_consensus.cc:686] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [term 1 LEADER]: Becoming Leader. State: Replica: f25341c9c79c45738bb5ccb2830c6022, State: Running, Role: LEADER
I20260806 03:52:01.248580 21280 consensus_queue.cc:237] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25341c9c79c45738bb5ccb2830c6022" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38891 } }
I20260806 03:52:01.249007 21281 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f25341c9c79c45738bb5ccb2830c6022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25341c9c79c45738bb5ccb2830c6022" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38891 } } }
I20260806 03:52:01.249003 21284 sys_catalog.cc:455] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f25341c9c79c45738bb5ccb2830c6022. Latest consensus state: current_term: 1 leader_uuid: "f25341c9c79c45738bb5ccb2830c6022" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f25341c9c79c45738bb5ccb2830c6022" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38891 } } }
I20260806 03:52:01.249145 21281 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:01.249148 21284 sys_catalog.cc:458] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:01.249711 21293 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:01.250291 21293 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:01.252072 21293 catalog_manager.cc:1347] Generated new cluster ID: fb7a7f9026d046be959c67e8d289e63f
I20260806 03:52:01.252126 21293 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:01.256568 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.246764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58810 (local address 127.12.15.126:38891)
0806 03:52:01.246991 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.247002 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.247035 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:52:01.247166 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.247174 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.247243 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.247388 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.247974 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.249481 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.250462 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.251081 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.251186 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.253551 (+ 2365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:01.253556 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:01.253556 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:01.253569 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:01.255159 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.255490 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.255494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.255495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.255597 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.255781 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.255782 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.255783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.256020 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:01.256178 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:01.256275 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 03:52:01.256412 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":123,"threads_started":1}
I20260806 03:52:01.257380 21293 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:52:01.257864 21293 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:52:01.264376 21293 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f25341c9c79c45738bb5ccb2830c6022: Generated new TSK 0
I20260806 03:52:01.264556 21293 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:52:01.299621 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38891
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:01.418609 21308 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:01.418817 21308 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:01.418840 21308 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:01.419971 21308 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:01.420013 21308 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:01.420038 21308 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:01.421504 21308 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21308
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:01.421717 21308 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:01.421918 21308 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:01.426936 21318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:01.426924 21322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:01.426923 21317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:01.427181 21308 server_base.cc:1034] running on GCE node
I20260806 03:52:01.427413 21308 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:01.427644 21308 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:01.428784 21308 hybrid_clock.cc:648] HybridClock initialized: now 1785988321428773 us; error 29 us; skew 500 ppm
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.431404 21308 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:52:01.431823 21308 webserver.cc:460] Webserver started at http://127.12.15.65:39715/ using document root <none> and password file <none>
I20260806 03:52:01.432037 21308 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:01.432082 21308 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:01.432201 21308 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:01.433205 21308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "799e9a9ecb234aabbd52db55b7744b5f"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "cdc6bebdd4310ebfc4efab267ab7f0b8"
server_key_iv: "fd95a3b38e893cfb6ca3a269512e11b0"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.433528 21308 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "799e9a9ecb234aabbd52db55b7744b5f"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "cdc6bebdd4310ebfc4efab267ab7f0b8"
server_key_iv: "fd95a3b38e893cfb6ca3a269512e11b0"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.434839 21308 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 03:52:01.435598 21330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.435782 21308 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:01.435868 21308 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "799e9a9ecb234aabbd52db55b7744b5f"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "cdc6bebdd4310ebfc4efab267ab7f0b8"
server_key_iv: "fd95a3b38e893cfb6ca3a269512e11b0"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.435925 21308 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:01.450228 21308 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:01.451951 21308 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:01.452078 21308 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:01.452275 21308 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:01.452564 21308 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:01.452600 21308 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.452622 21308 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:01.452633 21308 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.458240 21308 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:35743
I20260806 03:52:01.458307 21515 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:35743 every 8 connection(s)
I20260806 03:52:01.458595 21308 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:01.466562 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 21308
I20260806 03:52:01.466671 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:52:01.466867 12349 external_mini_cluster.cc:1428] Setting key e7ec9497fe1b2495eec5810c509dda92
I20260806 03:52:01.467844 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.459281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:40479 (local address 127.12.15.126:38891)
0806 03:52:01.459333 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.459335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.459565 (+ 230us) server_negotiation.cc:407] Connection header received
0806 03:52:01.460432 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.460432 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.460444 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.460504 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.461076 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.461591 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.462580 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.462778 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.462830 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.464964 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:01.464965 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:01.464965 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:01.464985 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:01.466256 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.466792 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.466793 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.466793 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.466856 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.467141 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.467143 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.467143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.467290 (+ 147us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:01.467360 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:01.467651 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 03:52:01.467751 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:52:01.468022 21522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.459243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38891 (local address 127.12.15.65:40479)
0806 03:52:01.459416 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:01.459457 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:01.460345 (+ 888us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:01.460560 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:01.460563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:01.460820 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:01.461010 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:01.461015 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.461655 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.461656 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:01.462514 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:01.462517 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.462835 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.462836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:01.462932 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.463357 (+ 425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:01.463365 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:01.464854 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:01.466339 (+ 1485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:01.466347 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:01.466349 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:01.466672 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:01.466923 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:01.466924 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:01.466924 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:01.467067 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:01.467360 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:01.467367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:01.467515 (+ 148us) client_negotiation.cc:769] Sending connection context
0806 03:52:01.467618 (+ 103us) client_negotiation.cc:240] Negotiation successful
0806 03:52:01.467802 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260806 03:52:01.468463 21516 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38891
I20260806 03:52:01.468556 21516 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:01.468741 21516 heartbeater.cc:511] Master 127.12.15.126:38891 requested a full tablet report, sending...
I20260806 03:52:01.469307 21223 ts_manager.cc:194] Registered new tserver with Master: 799e9a9ecb234aabbd52db55b7744b5f (127.12.15.65:35743)
I20260806 03:52:01.469926 21223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:40479
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:52:01.492312 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38891
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:01.573674 21525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:01.573875 21525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:01.573897 21525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:01.575083 21525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:01.575129 21525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:01.575153 21525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:01.576618 21525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:01.576808 21525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:01.576987 21525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:01.581789 21543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:01.581907 21525 server_base.cc:1034] running on GCE node
W20260806 03:52:01.581812 21538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:01.582023 21541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:01.582240 21525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:01.582451 21525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:01.583583 21525 hybrid_clock.cc:648] HybridClock initialized: now 1785988321583569 us; error 31 us; skew 500 ppm
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.585928 21525 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:52:01.586364 21525 webserver.cc:460] Webserver started at http://127.12.15.66:42845/ using document root <none> and password file <none>
I20260806 03:52:01.586562 21525 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:01.586601 21525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:01.586740 21525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:01.587744 21525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "d270d13d4191470ab9c642c7d73e5db8"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "4c2577b877cbef4ae4b06bcfdc5704c9"
server_key_iv: "34a503dbb5569f80e5be2814698bb7fb"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.588039 21525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "d270d13d4191470ab9c642c7d73e5db8"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "4c2577b877cbef4ae4b06bcfdc5704c9"
server_key_iv: "34a503dbb5569f80e5be2814698bb7fb"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.589182 21525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:52:01.589788 21559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.589936 21525 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.590001 21525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "d270d13d4191470ab9c642c7d73e5db8"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "4c2577b877cbef4ae4b06bcfdc5704c9"
server_key_iv: "34a503dbb5569f80e5be2814698bb7fb"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.590068 21525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:01.602452 21525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:01.604200 21525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:01.604321 21525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:01.604540 21525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:01.604839 21525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:01.604871 21525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.604892 21525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:01.604914 21525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.610391 21525 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:44891
I20260806 03:52:01.610462 21741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:44891 every 8 connection(s)
I20260806 03:52:01.610754 21525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:01.617177 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 21525
I20260806 03:52:01.617283 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:52:01.617475 12349 external_mini_cluster.cc:1428] Setting key 660f5d925de1c560ce9a41e5f67d2ee3
I20260806 03:52:01.619737 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.611447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:50059 (local address 127.12.15.126:38891)
0806 03:52:01.611532 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.611534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.611697 (+ 163us) server_negotiation.cc:407] Connection header received
0806 03:52:01.612609 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.612610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.612620 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.612667 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.613268 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.613756 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.614829 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.615011 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.615059 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.617060 (+ 2001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:01.617062 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:01.617062 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:01.617070 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:01.618486 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.618857 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.618858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.618859 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.618917 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.619128 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.619128 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.619129 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.619254 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:01.619308 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:01.619563 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 03:52:01.619647 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 03:52:01.619925 21749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.611414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38891 (local address 127.12.15.66:50059)
0806 03:52:01.611578 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:01.611616 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:01.612521 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:01.612733 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:01.612736 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:01.613003 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:01.613202 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:01.613207 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.613838 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.613839 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:01.614757 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:01.614760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.615066 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.615067 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:01.615160 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.615591 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:01.615600 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:01.616969 (+ 1369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:01.618558 (+ 1589us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:01.618565 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:01.618567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:01.618790 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:01.618968 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:01.618968 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:01.618969 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:01.619073 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:01.619308 (+ 235us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:01.619315 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:01.619447 (+ 132us) client_negotiation.cc:769] Sending connection context
0806 03:52:01.619536 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 03:52:01.619700 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":77,"threads_started":1}
I20260806 03:52:01.620350 21745 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38891
I20260806 03:52:01.620458 21745 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:01.620656 21745 heartbeater.cc:511] Master 127.12.15.126:38891 requested a full tablet report, sending...
I20260806 03:52:01.621025 21223 ts_manager.cc:194] Registered new tserver with Master: d270d13d4191470ab9c642c7d73e5db8 (127.12.15.66:44891)
I20260806 03:52:01.621366 21223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:50059
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:52:01.642582 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38891
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:01.724561 21753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:01.724781 21753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:01.724802 21753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:01.725908 21753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:01.725945 21753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:01.725968 21753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:01.727418 21753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:01.727612 21753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:01.727794 21753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:01.732614 21761 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:01.732685 21765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:01.732761 21753 server_base.cc:1034] running on GCE node
W20260806 03:52:01.732631 21760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:01.733019 21753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:01.733275 21753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:01.734418 21753 hybrid_clock.cc:648] HybridClock initialized: now 1785988321734404 us; error 26 us; skew 500 ppm
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.736891 21753 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:52:01.737342 21753 webserver.cc:460] Webserver started at http://127.12.15.67:37247/ using document root <none> and password file <none>
I20260806 03:52:01.737517 21753 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:01.737555 21753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:01.737656 21753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:01.738675 21753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "d75e97b217b346a68c180a9d07368c77"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "c359fbc6057a6eb8979975f557431ed9"
server_key_iv: "c38e7fd9fd49489b3029258ea1e7766d"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.739001 21753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "d75e97b217b346a68c180a9d07368c77"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "c359fbc6057a6eb8979975f557431ed9"
server_key_iv: "c38e7fd9fd49489b3029258ea1e7766d"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.740192 21753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:52:01.740813 21773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.740952 21753 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:01.741032 21753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "d75e97b217b346a68c180a9d07368c77"
format_stamp: "Formatted at 2026-08-06 03:52:01 on dist-test-slave-bqcl"
server_key: "c359fbc6057a6eb8979975f557431ed9"
server_key_iv: "c38e7fd9fd49489b3029258ea1e7766d"
server_key_version: "encryptionkey@0"
I20260806 03:52:01.741096 21753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:01.761024 21753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:01.762854 21753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:01.762982 21753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:01.763211 21753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:01.763527 21753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:01.763561 21753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.763582 21753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:01.763613 21753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:01.769299 21753 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:34615
I20260806 03:52:01.769354 21940 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:34615 every 8 connection(s)
I20260806 03:52:01.769667 21753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:01.777824 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 21753
I20260806 03:52:01.777913 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:52:01.778153 12349 external_mini_cluster.cc:1428] Setting key e973d1ec2f504492bdb35fdf7d6934f3
I20260806 03:52:01.779125 21279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.770459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:51437 (local address 127.12.15.126:38891)
0806 03:52:01.770511 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.770513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.770727 (+ 214us) server_negotiation.cc:407] Connection header received
0806 03:52:01.771648 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.771649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.771660 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.771720 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.772345 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.772857 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.773805 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.773999 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.774057 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.776168 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:01.776169 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:01.776169 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:01.776176 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:01.777519 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.778042 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.778044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.778044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.778105 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.778398 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.778400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.778400 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.778542 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:01.778607 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:01.778933 (+ 326us) server_negotiation.cc:299] Negotiation successful
0806 03:52:01.779018 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:52:01.779282 21945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.770443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38891 (local address 127.12.15.67:51437)
0806 03:52:01.770601 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:01.770638 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:01.771530 (+ 892us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:01.771790 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:01.771794 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:01.772059 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:01.772260 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:01.772265 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.772923 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.772924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:01.773722 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:01.773725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.774063 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.774064 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:01.774199 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.774677 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:01.774685 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:01.776082 (+ 1397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:01.777605 (+ 1523us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:01.777612 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:01.777614 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:01.777925 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:01.778189 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:01.778190 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:01.778190 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:01.778328 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:01.778605 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:01.778611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:01.778787 (+ 176us) client_negotiation.cc:769] Sending connection context
0806 03:52:01.778892 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 03:52:01.779053 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":84,"threads_started":1}
I20260806 03:52:01.779737 21943 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38891
I20260806 03:52:01.779824 21943 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:01.779994 21943 heartbeater.cc:511] Master 127.12.15.126:38891 requested a full tablet report, sending...
I20260806 03:52:01.780323 21223 ts_manager.cc:194] Registered new tserver with Master: d75e97b217b346a68c180a9d07368c77 (127.12.15.67:34615)
I20260806 03:52:01.780686 21223 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:51437
I20260806 03:52:01.790009 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/test-admin.keytab.
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.806612 12349 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260806 03:52:01.808276 21952 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:01.807519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.12.15.65:35743)
0806 03:52:01.807710 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.807715 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.807723 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:01.807748 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.807750 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.807763 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.807828 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.808191 (+ 363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48350: BlockingRecv error: recv got EOF from 127.0.0.1:48350 (error 108)
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":95,"threads_started":1}
W20260806 03:52:01.808354 12349 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5cc)
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.809340 12349 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.810240 12349 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:52:01.817420 21952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.810753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.12.15.65:35743)
0806 03:52:01.810811 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.810812 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.810853 (+ 41us) server_negotiation.cc:407] Connection header received
0806 03:52:01.810903 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.810904 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.810914 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.810959 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.811305 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.812021 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.812719 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.813020 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.813067 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.815059 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:01.815067 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:01.815068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:01.815075 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:01.816411 (+ 1336us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.816726 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.816728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.816729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.816781 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.816922 (+ 141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.816922 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.816923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.817038 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:01.817146 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:01.817260 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 03:52:01.817322 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
W20260806 03:52:01.819032 21952 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:01.818612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48378 (local address 127.12.15.65:35743)
0806 03:52:01.818647 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.818648 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.818668 (+ 20us) server_negotiation.cc:407] Connection header received
0806 03:52:01.818723 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.818724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.818732 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.818765 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.818986 (+ 221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48378: BlockingRecv error: recv got EOF from 127.0.0.1:48378 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 03:52:01.819108 12349 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5cc' of type FILE
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.819950 12349 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.820708 12349 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:52:01.826764 21952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.821194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48384 (local address 127.12.15.65:35743)
0806 03:52:01.821237 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.821238 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.821247 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:01.821284 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.821285 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.821292 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.821333 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.821634 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.822232 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.822879 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.823090 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.823149 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.824934 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:01.824935 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:01.824936 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:01.824942 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:01.825963 (+ 1021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.826285 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.826286 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.826286 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.826332 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.826459 (+ 127us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.826460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.826460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.826532 (+ 72us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:01.826593 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:01.826653 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 03:52:01.826710 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
W20260806 03:52:01.828112 21952 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:01.827623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48400 (local address 127.12.15.65:35743)
0806 03:52:01.827659 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.827661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.827717 (+ 56us) server_negotiation.cc:407] Connection header received
0806 03:52:01.827781 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.827782 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.827790 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.827834 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.828079 (+ 245us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48400: BlockingRecv error: recv got EOF from 127.0.0.1:48400 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 03:52:01.828346 12349 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1785988295328500-12349-0/krb5kdc/krb5cc' of type FILE
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.829206 12349 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:01.830049 12349 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988321, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:52:01.836422 21952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:01.830507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.12.15.65:35743)
0806 03:52:01.830559 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:01.830560 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:01.830568 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:01.830619 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:01.830620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:01.830628 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:01.830672 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:01.830996 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.831449 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.832079 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:01.832276 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:01.832328 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:01.834110 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:01.834112 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:01.834112 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:01.834119 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:01.835422 (+ 1303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.835742 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.835743 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.835743 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.835797 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:01.835957 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:01.835958 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:01.835958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:01.836061 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:01.836142 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:01.836257 (+ 115us) server_negotiation.cc:299] Negotiation successful
0806 03:52:01.836329 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:52:01.837098 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 21308
I20260806 03:52:01.854430 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 21525
I20260806 03:52:01.867941 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 21753
I20260806 03:52:01.880398 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 21175
2026-08-06T03:52:01Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2007 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21970](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21970](info): set up 2 sockets
Aug 06 03:52:01 dist-test-slave-bqcl krb5kdc[21970](info): commencing operation
krb5kdc: starting...
W20260806 03:52:03.021449 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:52:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.126.keytab.
Entry for principal oryx/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.126.keytab.
I20260806 03:52:03.139226 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:40955
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.126.keytab
--principal=oryx/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:40955
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:03.221467 21990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:03.221688 21990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:03.221710 21990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:03.222940 21990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:03.222996 21990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:03.223011 21990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:03.223035 21990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:03.223059 21990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:03.224591 21990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:40955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.126.keytab
--principal=oryx/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:40955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:03.224836 21990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:03.225063 21990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:03.229655 22004 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.229661 22007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.229679 22002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:03.229954 21990 server_base.cc:1034] running on GCE node
I20260806 03:52:03.230269 21990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:03.230544 21990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:03.231700 21990 hybrid_clock.cc:648] HybridClock initialized: now 1785988323231686 us; error 29 us; skew 500 ppm
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:03.237668 21990 init.cc:377] Logged in from keytab as oryx/127.12.15.126@KRBTEST.COM (short username oryx)
I20260806 03:52:03.238059 21990 webserver.cc:460] Webserver started at http://127.12.15.126:32809/ using document root <none> and password file <none>
I20260806 03:52:03.238269 21990 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:03.238315 21990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:03.238410 21990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:03.239389 21990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "2dcfdad11d234a31a0a5a49e42fbda99"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "754add358ae53170d743aac9ebb15234"
server_key_iv: "06fbfa98a785672424026ad49e21db89"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.239674 21990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "2dcfdad11d234a31a0a5a49e42fbda99"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "754add358ae53170d743aac9ebb15234"
server_key_iv: "06fbfa98a785672424026ad49e21db89"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.240832 21990 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:52:03.241453 22020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.241647 21990 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:03.241734 21990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "2dcfdad11d234a31a0a5a49e42fbda99"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "754add358ae53170d743aac9ebb15234"
server_key_iv: "06fbfa98a785672424026ad49e21db89"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.241791 21990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:03.259048 21990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:03.260891 21990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:03.261009 21990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:03.264810 21990 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:40955
I20260806 03:52:03.264853 22103 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:40955 every 8 connection(s)
I20260806 03:52:03.265182 21990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:52:03.266208 22104 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:03.268455 22104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: Bootstrap starting.
I20260806 03:52:03.269006 22104 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:03.269277 22104 log.cc:824] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:03.269932 22104 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: No bootstrap required, opened a new log
I20260806 03:52:03.271499 22104 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } }
I20260806 03:52:03.271632 22104 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:03.271654 22104 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2dcfdad11d234a31a0a5a49e42fbda99, State: Initialized, Role: FOLLOWER
I20260806 03:52:03.271792 22104 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } }
I20260806 03:52:03.271858 22104 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:52:03.271890 22104 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:52:03.271937 22104 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:03.272682 22104 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } }
I20260806 03:52:03.272778 22104 leader_election.cc:302] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2dcfdad11d234a31a0a5a49e42fbda99; no voters:
I20260806 03:52:03.272926 22104 leader_election.cc:288] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:52:03.272961 22107 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:03.273097 22107 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 1 LEADER]: Becoming Leader. State: Replica: 2dcfdad11d234a31a0a5a49e42fbda99, State: Running, Role: LEADER
I20260806 03:52:03.273172 22104 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:03.273185 22107 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } }
I20260806 03:52:03.273564 22109 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dcfdad11d234a31a0a5a49e42fbda99. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } }
I20260806 03:52:03.273576 22108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } }
I20260806 03:52:03.273723 22108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:03.274062 22109 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:03.274099 22119 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:03.274554 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 21990
I20260806 03:52:03.274658 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:52:03.274694 22119 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:03.274855 12349 external_mini_cluster.cc:1428] Setting key 5f60f71fa0cf1b5afd6980e3c19b781e
I20260806 03:52:03.276314 22119 catalog_manager.cc:1347] Generated new cluster ID: 2608214565024160a23a9ecdf1abeae3
I20260806 03:52:03.276366 22119 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:52:03.285967 22119 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:52:03.286520 22119 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:03.292356 22119 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: Generated new TSK 0
I20260806 03:52:03.292557 22119 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:52:03.295214 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.275297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58234 (local address 127.12.15.126:40955)
0806 03:52:03.275453 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.275462 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.275854 (+ 392us) server_negotiation.cc:407] Connection header received
0806 03:52:03.276021 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.276029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.276096 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.276205 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.285991 (+ 9786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.287505 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.288594 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.289213 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.289325 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.292130 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.292134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.292134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.292147 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.293718 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.294110 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.294112 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.294112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.294242 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.294428 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.294428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.294428 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.294624 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.294749 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.294878 (+ 129us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.294990 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for oryx/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab.
Entry for principal oryx/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab.
I20260806 03:52:03.320916 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40955
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab
--principal=oryx/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:03.402683 22137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:03.402884 22137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:03.402905 22137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:03.404022 22137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:03.404070 22137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:03.404099 22137 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:03.405565 22137 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab
--principal=oryx/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:03.405787 22137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:03.406025 22137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:03.410864 22143 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:03.411022 22137 server_base.cc:1034] running on GCE node
W20260806 03:52:03.411079 22144 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.411079 22154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:03.411423 22137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:03.411679 22137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:03.412829 22137 hybrid_clock.cc:648] HybridClock initialized: now 1785988323412818 us; error 27 us; skew 500 ppm
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:03.415243 22137 init.cc:377] Logged in from keytab as oryx/127.12.15.65@KRBTEST.COM (short username oryx)
I20260806 03:52:03.415699 22137 webserver.cc:460] Webserver started at http://127.12.15.65:36689/ using document root <none> and password file <none>
I20260806 03:52:03.415890 22137 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:03.415938 22137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:03.416064 22137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:03.417058 22137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "34bcb6df749444de99b1fe01c02d6c6a"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "22bb46d529e37aa37fef1f1373bfa9d1"
server_key_iv: "894c404ed16da85b02abb9751a6cb708"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.417383 22137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "34bcb6df749444de99b1fe01c02d6c6a"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "22bb46d529e37aa37fef1f1373bfa9d1"
server_key_iv: "894c404ed16da85b02abb9751a6cb708"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.418653 22137 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:03.419301 22164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.419449 22137 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:03.419533 22137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "34bcb6df749444de99b1fe01c02d6c6a"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "22bb46d529e37aa37fef1f1373bfa9d1"
server_key_iv: "894c404ed16da85b02abb9751a6cb708"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.419596 22137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:03.433538 22137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:03.435286 22137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:03.435422 22137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:03.435668 22137 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:03.436021 22137 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:03.436100 22137 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.436161 22137 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:03.436196 22137 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.441656 22137 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:43879
I20260806 03:52:03.441702 22333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:43879 every 8 connection(s)
I20260806 03:52:03.442010 22137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:52:03.446103 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 22137
I20260806 03:52:03.446223 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:52:03.446429 12349 external_mini_cluster.cc:1428] Setting key 08916cff03c9508955c53539599583fb
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:03.451341 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.442717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:38313 (local address 127.12.15.126:40955)
0806 03:52:03.442778 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.442780 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.442983 (+ 203us) server_negotiation.cc:407] Connection header received
0806 03:52:03.443475 (+ 492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.443476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.443486 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.443542 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.444462 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.445009 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.445973 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.446253 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.446314 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.448656 (+ 2342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.448658 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.448658 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.448664 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.449934 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.450423 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.450424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.450424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.450485 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.450744 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.450744 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.450745 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.450849 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.450896 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.451204 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.451277 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:52:03.451586 22336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.442701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:38313)
0806 03:52:03.442860 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.442904 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.443391 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.443597 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.443600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.444161 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.444385 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.444391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.445088 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.445089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.445898 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.445901 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.446347 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.446348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.446498 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.447000 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.447010 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.448571 (+ 1561us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.450013 (+ 1442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.450021 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.450024 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.450352 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.450545 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.450546 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.450546 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.450681 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.450901 (+ 220us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.450908 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.451074 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.451180 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.451338 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
I20260806 03:52:03.452132 22334 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40955
I20260806 03:52:03.452229 22334 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:03.452397 22334 heartbeater.cc:511] Master 127.12.15.126:40955 requested a full tablet report, sending...
I20260806 03:52:03.452917 22051 ts_manager.cc:194] Registered new tserver with Master: 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65:43879)
I20260806 03:52:03.453528 22051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:38313
WARNING: no policy specified for oryx/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab.
Entry for principal oryx/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab.
I20260806 03:52:03.472553 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40955
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab
--principal=oryx/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:03.552385 22339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:03.552587 22339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:03.552618 22339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:03.553719 22339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:03.553759 22339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:03.553793 22339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:03.555248 22339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab
--principal=oryx/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:03.555480 22339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:03.555692 22339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:03.560488 22348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.560524 22351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.560603 22349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:03.560624 22339 server_base.cc:1034] running on GCE node
I20260806 03:52:03.560977 22339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:03.561218 22339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:03.562366 22339 hybrid_clock.cc:648] HybridClock initialized: now 1785988323562351 us; error 28 us; skew 500 ppm
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:03.564824 22339 init.cc:377] Logged in from keytab as oryx/127.12.15.66@KRBTEST.COM (short username oryx)
I20260806 03:52:03.565235 22339 webserver.cc:460] Webserver started at http://127.12.15.66:44187/ using document root <none> and password file <none>
I20260806 03:52:03.565407 22339 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:03.565445 22339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:03.565537 22339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:03.566537 22339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "c7af4122ee524f78a548e4f2be47996a"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "1edc660724e20506c9f17c76004bb75c"
server_key_iv: "c1fb7fb4c50bab19675d3e8367313f55"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.566833 22339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "c7af4122ee524f78a548e4f2be47996a"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "1edc660724e20506c9f17c76004bb75c"
server_key_iv: "c1fb7fb4c50bab19675d3e8367313f55"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.568041 22339 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:52:03.568656 22365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.568825 22339 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:03.568900 22339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "c7af4122ee524f78a548e4f2be47996a"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "1edc660724e20506c9f17c76004bb75c"
server_key_iv: "c1fb7fb4c50bab19675d3e8367313f55"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.568965 22339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:03.580693 22339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:03.582566 22339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:03.582679 22339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:03.582895 22339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:03.583177 22339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:03.583218 22339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.583238 22339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:03.583261 22339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.588753 22339 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:40675
I20260806 03:52:03.588814 22547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:40675 every 8 connection(s)
I20260806 03:52:03.589077 22339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:03.597307 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.589763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:44843 (local address 127.12.15.126:40955)
0806 03:52:03.589843 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.589845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.589989 (+ 144us) server_negotiation.cc:407] Connection header received
0806 03:52:03.590488 (+ 499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.590489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.590505 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.590562 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.591442 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.591942 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.592849 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.593037 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.593090 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.595037 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.595039 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.595039 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.595046 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.596207 (+ 1161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.596574 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.596574 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.596575 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.596620 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.596790 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.596791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.596791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.596883 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.596926 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.597135 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.597214 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:52:03.597450 22551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.589727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:44843)
0806 03:52:03.589880 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.589915 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.590409 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.590622 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.590625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.591178 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.591363 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.591368 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.592008 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.592008 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.592784 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.592786 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.593092 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.593093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.593186 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.593627 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.593634 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.594956 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.596282 (+ 1326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.596289 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.596291 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.596512 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.596661 (+ 149us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.596661 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.596661 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.596750 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.596923 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.596926 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.597044 (+ 118us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.597121 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.597238 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":82,"threads_started":1}
I20260806 03:52:03.597497 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 22339
I20260806 03:52:03.597637 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:52:03.597844 12349 external_mini_cluster.cc:1428] Setting key 34f64c2d0ec82f2ce3db565c2a619d76
I20260806 03:52:03.597950 22548 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40955
I20260806 03:52:03.598047 22548 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:03.598258 22548 heartbeater.cc:511] Master 127.12.15.126:40955 requested a full tablet report, sending...
I20260806 03:52:03.598598 22051 ts_manager.cc:194] Registered new tserver with Master: c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675)
I20260806 03:52:03.598955 22051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:44843
WARNING: no policy specified for oryx/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab.
Entry for principal oryx/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab.
I20260806 03:52:03.622335 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40955
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab
--principal=oryx/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:03.703600 22555 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:03.703799 22555 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:03.703822 22555 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:03.704882 22555 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:03.704916 22555 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:03.704939 22555 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:03.706404 22555 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab
--principal=oryx/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:40955
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22555
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:03.706633 22555 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:03.706816 22555 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:03.711570 22568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.711578 22570 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.711603 22564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:03.711658 22555 server_base.cc:1034] running on GCE node
I20260806 03:52:03.712052 22555 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:03.712301 22555 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:03.713435 22555 hybrid_clock.cc:648] HybridClock initialized: now 1785988323713422 us; error 27 us; skew 500 ppm
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:03.715752 22555 init.cc:377] Logged in from keytab as oryx/127.12.15.67@KRBTEST.COM (short username oryx)
I20260806 03:52:03.716142 22555 webserver.cc:460] Webserver started at http://127.12.15.67:43991/ using document root <none> and password file <none>
I20260806 03:52:03.716338 22555 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:03.716385 22555 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:03.716508 22555 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:03.717549 22555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "7f113aabda2943449519b0a57873f4ce"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "6f439daa3b2ad355485a8144e6f9ddf5"
server_key_iv: "dbd1367fba801aa3702af739cf86488b"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.717862 22555 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "7f113aabda2943449519b0a57873f4ce"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "6f439daa3b2ad355485a8144e6f9ddf5"
server_key_iv: "dbd1367fba801aa3702af739cf86488b"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.719092 22555 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:03.719741 22581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.719892 22555 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:03.719973 22555 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "7f113aabda2943449519b0a57873f4ce"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "6f439daa3b2ad355485a8144e6f9ddf5"
server_key_iv: "dbd1367fba801aa3702af739cf86488b"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.720034 22555 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:03.744025 22555 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:03.745782 22555 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:03.745918 22555 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:03.746147 22555 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:03.746461 22555 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:03.746495 22555 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.746522 22555 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:03.746546 22555 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.752065 22555 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:45395
I20260806 03:52:03.752130 22771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:45395 every 8 connection(s)
I20260806 03:52:03.752409 22555 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:52:03.757231 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 22555
I20260806 03:52:03.757341 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:52:03.757576 12349 external_mini_cluster.cc:1428] Setting key 4569b7801100f97f6270ab6eccd3f7df
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:03.761447 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.753084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:60571 (local address 127.12.15.126:40955)
0806 03:52:03.753135 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.753137 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.753300 (+ 163us) server_negotiation.cc:407] Connection header received
0806 03:52:03.753837 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.753839 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.753853 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.753902 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.754935 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.755441 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.756491 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.756681 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.756737 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.759155 (+ 2418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.759156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.759157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.759164 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.760284 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.760650 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.760652 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.760652 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.760721 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.760908 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.760908 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.760908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.761014 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.761060 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.761306 (+ 246us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.761378 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:52:03.761514 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.753047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:60571)
0806 03:52:03.753184 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.753225 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.753736 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.753982 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.753986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.754647 (+ 661us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.754838 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.754842 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.755534 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.755536 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.756411 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.756414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.756750 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.756751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.756881 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.757405 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.757416 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.759040 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.760352 (+ 1312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.760358 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.760360 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.760565 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.760777 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.760777 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.760777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.760865 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.761055 (+ 190us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.761058 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.761166 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.761242 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.761355 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":66,"threads_started":1}
I20260806 03:52:03.761926 22772 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40955
I20260806 03:52:03.762014 22772 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:03.762229 22772 heartbeater.cc:511] Master 127.12.15.126:40955 requested a full tablet report, sending...
I20260806 03:52:03.762558 22051 ts_manager.cc:194] Registered new tserver with Master: 7f113aabda2943449519b0a57873f4ce (127.12.15.67:45395)
I20260806 03:52:03.762913 22051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:60571
I20260806 03:52:03.769239 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.15.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.125.keytab.
Entry for principal oryx/127.12.15.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.125.keytab.
I20260806 03:52:03.793234 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.125:42223
--webserver_interface=127.12.15.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.125.keytab
--principal=oryx/127.12.15.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:40955,127.12.15.125:42223
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:03.874056 22779 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:03.874310 22779 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:03.874336 22779 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:03.875478 22779 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:03.875526 22779 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:03.875540 22779 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:03.875551 22779 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:03.875576 22779 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:03.877048 22779 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:40955,127.12.15.125:42223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.125.keytab
--principal=oryx/127.12.15.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.125:42223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.15.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22779
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
W20260806 03:52:03.877166 22779 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.12.15.126:40955,127.12.15.125:42223'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260806 03:52:03.877311 22779 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:03.877538 22779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:03.882191 22786 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.882207 22785 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:03.882232 22779 server_base.cc:1034] running on GCE node
W20260806 03:52:03.882241 22789 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:03.882543 22779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:03.882819 22779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:03.883956 22779 hybrid_clock.cc:648] HybridClock initialized: now 1785988323883944 us; error 27 us; skew 500 ppm
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:03.886407 22779 init.cc:377] Logged in from keytab as oryx/127.12.15.125@KRBTEST.COM (short username oryx)
I20260806 03:52:03.886808 22779 webserver.cc:460] Webserver started at http://127.12.15.125:46749/ using document root <none> and password file <none>
I20260806 03:52:03.887007 22779 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:03.887054 22779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:03.887197 22779 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:03.888182 22779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data/instance:
uuid: "87612679f1764898a407f02fbd6a5c94"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "583c112b0c5bfc3035e599332e8170a5"
server_key_iv: "0c140755ce2fce42d673964b8b4906fa"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.888484 22779 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal/instance:
uuid: "87612679f1764898a407f02fbd6a5c94"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "583c112b0c5bfc3035e599332e8170a5"
server_key_iv: "0c140755ce2fce42d673964b8b4906fa"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.889667 22779 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:03.890326 22799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.890475 22779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:03.890551 22779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
uuid: "87612679f1764898a407f02fbd6a5c94"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "583c112b0c5bfc3035e599332e8170a5"
server_key_iv: "0c140755ce2fce42d673964b8b4906fa"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.890617 22779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:03.900396 22779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:03.902392 22779 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:03.902513 22779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:03.906075 22779 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.125:42223
I20260806 03:52:03.906121 22875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.125:42223 every 8 connection(s)
I20260806 03:52:03.906471 22779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
I20260806 03:52:03.907439 22877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:03.907545 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 22779
I20260806 03:52:03.909411 22877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.125@KRBTEST.COM
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.125@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:03.917920 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.910017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58240 (local address 127.12.15.126:40955)
0806 03:52:03.910075 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.910077 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.910207 (+ 130us) server_negotiation.cc:407] Connection header received
0806 03:52:03.910282 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.910283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.910292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.910340 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.910878 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.911671 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.912725 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.912906 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.912949 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.915513 (+ 2564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.915515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.915515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.915523 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.916836 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.917142 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.917144 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.917144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.917200 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.917378 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.917378 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.917379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.917522 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.917578 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.917752 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.917827 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:52:03.918098 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.908174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37022 (local address 127.12.15.125:42223)
0806 03:52:03.908324 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.908333 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.908393 (+ 60us) server_negotiation.cc:407] Connection header received
0806 03:52:03.908858 (+ 465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.908864 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.908910 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.909027 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.909609 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.910821 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.911825 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.912401 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.912503 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.914886 (+ 2383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.914894 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.914895 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.914902 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.916577 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.916959 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.916962 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.916963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.917055 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.917234 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.917235 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.917235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.917456 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.917608 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.917720 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.917877 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":102,"thread_start_us":56,"threads_started":1}
I20260806 03:52:03.918349 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.909943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58240)
0806 03:52:03.910089 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.910159 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.910225 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.910357 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.910359 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.910674 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.910788 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.910794 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.911759 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.911760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.912633 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.912635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.912981 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.912982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.913061 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.913460 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.913470 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.915433 (+ 1963us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.916899 (+ 1466us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.916905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.916907 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.917077 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.917254 (+ 177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.917255 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.917255 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.917323 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.917581 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.917587 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.917647 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.917720 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.918255 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":384,"thread_start_us":80,"threads_started":1}
I20260806 03:52:03.918920 22877 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.125@KRBTEST.COM for oryx/127.12.15.125@KRBTEST.COM
I20260806 03:52:03.924499 22885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.919144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.0.0.1:37036)
0806 03:52:03.919189 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.919216 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.919253 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.919356 (+ 103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.919356 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.919460 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.919488 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.919489 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.920019 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.920019 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.920477 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.920478 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.920783 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.920784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.920828 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.921115 (+ 287us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.921118 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.922403 (+ 1285us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.923763 (+ 1360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.923764 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.923764 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.923916 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.924073 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.924073 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.924074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.924121 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.924289 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.924290 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.924322 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.924368 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.924440 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 03:52:03.924585 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.919200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37036 (local address 127.12.15.125:42223)
0806 03:52:03.919234 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.919235 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.919243 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:03.919296 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.919297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.919303 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.919343 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.919532 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.919956 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.920541 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.920731 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.920783 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.922527 (+ 1744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.922528 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.922529 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.922536 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.923696 (+ 1160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.923974 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.923974 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.923975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.924027 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.924155 (+ 128us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.924156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.924156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.924243 (+ 87us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.924291 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.924386 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.924492 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":18}
I20260806 03:52:03.925818 22877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Bootstrap starting.
I20260806 03:52:03.926661 22877 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:03.926901 22877 log.cc:824] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:03.927611 22877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: No bootstrap required, opened a new log
I20260806 03:52:03.928941 22877 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } }
I20260806 03:52:03.929067 22877 raft_consensus.cc:374] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:03.929096 22877 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87612679f1764898a407f02fbd6a5c94, State: Initialized, Role: FOLLOWER
I20260806 03:52:03.929181 22877 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } }
I20260806 03:52:03.929422 22877 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:03.929394 22888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } } }
I20260806 03:52:03.929473 22888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:52:03.931017 22906 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:52:03.931072 22906 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:52:03.932658 22779 master_runner.cc:428] Detected that this master 87612679f1764898a407f02fbd6a5c94 is joining an existing cluster
I20260806 03:52:03.932734 22779 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.15.125:42223
I20260806 03:52:03.939886 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.933830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.12.15.126:40955)
0806 03:52:03.933872 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.933874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.934021 (+ 147us) server_negotiation.cc:407] Connection header received
0806 03:52:03.934109 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.934110 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.934141 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.934195 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.934562 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.935262 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.935982 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.936277 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.936337 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.937308 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.937309 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.937310 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.937316 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.938711 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.939138 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.939140 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.939140 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.939201 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.939395 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.939396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.939397 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.939521 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.939554 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.939746 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.939821 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:52:03.940201 22915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.933650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58244)
0806 03:52:03.933960 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.933968 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.934044 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.934199 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.934200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.934426 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.934484 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.934486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.935322 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.935323 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.935911 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.935913 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.936353 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.936354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.936417 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.936799 (+ 382us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.936803 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.937222 (+ 419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.938786 (+ 1564us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.938787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.938788 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.939072 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.939272 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.939272 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.939273 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.939325 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.939592 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.939592 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.939634 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.939996 (+ 362us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.940094 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":76,"threads_started":1}
I20260806 03:52:03.940685 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.933807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37044 (local address 127.12.15.125:42223)
0806 03:52:03.933839 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.933840 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.933853 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:52:03.933933 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.933933 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.933942 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.933992 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.934405 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.935083 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.936071 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.936330 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.936380 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.937333 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.937334 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.937334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.937340 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.939134 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.939452 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.939453 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.939454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.939559 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.939751 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.939752 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.939752 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.939879 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.939943 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.940471 (+ 528us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.940589 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":17}
I20260806 03:52:03.940798 22913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.933635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.0.0.1:37044)
0806 03:52:03.933800 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.933812 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.933872 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.934006 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.934007 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.934261 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.934342 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.934344 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.935171 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.935172 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.935990 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.935993 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.936419 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.936420 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.936465 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.936912 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.936917 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.937239 (+ 322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.939205 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.939206 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.939207 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.939389 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.939634 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.939634 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.939634 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.939695 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.940169 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.940170 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.940210 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.940441 (+ 231us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.940541 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":1,"thread_start_us":86,"threads_started":1}
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.126@KRBTEST.COM for oryx/127.12.15.125@KRBTEST.COM
I20260806 03:52:03.949401 22884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.942616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37052 (local address 127.12.15.125:42223)
0806 03:52:03.942669 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.942671 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.942814 (+ 143us) server_negotiation.cc:407] Connection header received
0806 03:52:03.942902 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.942902 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.942910 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.942952 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.943445 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.943907 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.944967 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.945171 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.945240 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.947250 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.947252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.947252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.947259 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.948382 (+ 1123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.948714 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.948715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.948716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.948765 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.948926 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.948927 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.948927 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.949017 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.949069 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.949239 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.949311 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:52:03.949409 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.942559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.0.0.1:37052)
0806 03:52:03.942710 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.942754 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.942829 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.942958 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.942960 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.943299 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.943354 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.943356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.943972 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.943973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.944899 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.944901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.945251 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.945252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.945340 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.945770 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.945775 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.947173 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.948455 (+ 1282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.948463 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.948465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.948650 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.948819 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.948820 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.948820 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.948877 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.949071 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.949080 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.949131 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.949202 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.949314 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":95,"threads_started":1}
I20260806 03:52:03.949918 22051 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.15.125:42223: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "87612679f1764898a407f02fbd6a5c94"
member_type: NON_VOTER
last_known_addr {
host: "127.12.15.125"
port: 42223
}
attrs {
promote: true
}
}
dest_uuid: "2dcfdad11d234a31a0a5a49e42fbda99"
cas_config_opid_index: -1
I20260806 03:52:03.950335 22051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } }
I20260806 03:52:03.950850 22920 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 87612679f1764898a407f02fbd6a5c94 (127.12.15.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } }
I20260806 03:52:03.951114 22840 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:03.951853 22840 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Refusing update from remote peer 2dcfdad11d234a31a0a5a49e42fbda99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260806 03:52:03.951902 22920 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.15.125:42223
I20260806 03:52:03.952220 22779 master.cc:537] Master@127.12.15.125:42223 shutting down...
I20260806 03:52:03.952207 22921 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:03.952229 22919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } }
I20260806 03:52:03.952311 22919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
W20260806 03:52:03.952913 22028 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260806 03:52:03.953316 22028 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 -> Peer 87612679f1764898a407f02fbd6a5c94 (127.12.15.125:42223): Couldn't send request to peer 87612679f1764898a407f02fbd6a5c94. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:52:03.954922 22779 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:03.955025 22779 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:03.955081 22779 tablet_replica.cc:330] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: stopping tablet replica
I20260806 03:52:03.967893 22779 master.cc:552] Master@127.12.15.125:42223 shutdown complete.
I20260806 03:52:03.967952 22779 master_runner.cc:305] Clearing existing system tablet
I20260806 03:52:03.968087 22779 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:03.968124 22779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:03.968713 22779 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.001s
I20260806 03:52:03.969164 22928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.969278 22779 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:03.969355 22779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.969383 22779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
uuid: "87612679f1764898a407f02fbd6a5c94"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "583c112b0c5bfc3035e599332e8170a5"
server_key_iv: "0c140755ce2fce42d673964b8b4906fa"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.970323 22779 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:03.971774 22779 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 03:52:03.971846 22779 log.cc:1196] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260806 03:52:03.972046 22779 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Deleting consensus metadata
I20260806 03:52:03.972158 22779 master_runner.cc:315] Copying system tablet from 127.12.15.126:40955
I20260806 03:52:03.972759 22779 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: tablet copy: Beginning tablet copy session from remote peer at address 127.12.15.126:40955
I20260806 03:52:03.978976 22935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.973065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58254)
0806 03:52:03.973205 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:03.973215 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:03.973270 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:03.973397 (+ 127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:03.973398 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:03.973607 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:03.973673 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.973675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.974705 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.974706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.975267 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:03.975269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.975712 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.975713 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:03.975757 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.976215 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:03.976219 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:03.976493 (+ 274us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:03.978086 (+ 1593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.978087 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.978087 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.978277 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.978459 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:03.978459 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:03.978460 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:03.978507 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:03.978737 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:03.978738 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:03.978772 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 03:52:03.978831 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 03:52:03.978905 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":61,"threads_started":1}
I20260806 03:52:03.979029 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:03.973138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58254 (local address 127.12.15.126:40955)
0806 03:52:03.973180 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:03.973182 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:03.973274 (+ 92us) server_negotiation.cc:407] Connection header received
0806 03:52:03.973307 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:03.973308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:03.973319 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:03.973389 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:03.973749 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.974633 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.975338 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:03.975646 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:03.975714 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:03.976571 (+ 857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:03.976573 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:03.976573 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:03.976581 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:03.978006 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.978339 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.978340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.978340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.978403 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:03.978557 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:03.978558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:03.978558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:03.978685 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:03.978750 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:03.978854 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 03:52:03.978927 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:52:03.979516 22081 tablet_copy_service.cc:140] P 2dcfdad11d234a31a0a5a49e42fbda99: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 87612679f1764898a407f02fbd6a5c94 ({username='oryx', principal='oryx/127.12.15.125@KRBTEST.COM'} at 127.0.0.1:58254)
I20260806 03:52:03.979635 22081 tablet_copy_service.cc:161] P 2dcfdad11d234a31a0a5a49e42fbda99: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 87612679f1764898a407f02fbd6a5c94 at {username='oryx', principal='oryx/127.12.15.125@KRBTEST.COM'} at 127.0.0.1:58254: session id = 87612679f1764898a407f02fbd6a5c94-00000000000000000000000000000000
I20260806 03:52:03.980338 22081 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 03:52:03.980939 22779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 03:52:03.982296 22779 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: tablet copy: Starting download of 0 data blocks...
I20260806 03:52:03.982435 22779 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: tablet copy: Starting download of 1 WAL segments...
I20260806 03:52:03.983321 22779 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 03:52:03.984248 22081 tablet_copy_service.cc:342] P 2dcfdad11d234a31a0a5a49e42fbda99: Request end of tablet copy session 87612679f1764898a407f02fbd6a5c94-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.15.125@KRBTEST.COM'} at 127.0.0.1:58254
I20260806 03:52:03.984297 22081 tablet_copy_service.cc:433] P 2dcfdad11d234a31a0a5a49e42fbda99: ending tablet copy session 87612679f1764898a407f02fbd6a5c94-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 87612679f1764898a407f02fbd6a5c94
I20260806 03:52:03.986905 22779 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:03.988034 22936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.988006 22937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:03.988039 22939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:03.988345 22779 server_base.cc:1034] running on GCE node
I20260806 03:52:03.988427 22779 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:03.988554 22779 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:03.989665 22779 hybrid_clock.cc:648] HybridClock initialized: now 1785988323989659 us; error 23 us; skew 500 ppm
Aug 06 03:52:03 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:03.990926 22779 init.cc:377] Logged in from keytab as oryx/127.12.15.125@KRBTEST.COM (short username oryx)
I20260806 03:52:03.991178 22779 webserver.cc:460] Webserver started at http://127.12.15.125:33133/ using document root <none> and password file <none>
I20260806 03:52:03.991302 22779 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:03.991336 22779 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:03.991866 22779 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.992332 22947 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.992455 22779 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:03.992496 22779 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
uuid: "87612679f1764898a407f02fbd6a5c94"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "583c112b0c5bfc3035e599332e8170a5"
server_key_iv: "0c140755ce2fce42d673964b8b4906fa"
server_key_version: "encryptionkey@0"
I20260806 03:52:03.992547 22779 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:04.012980 22779 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:04.014536 22779 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:04.017361 22779 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.125:42223
I20260806 03:52:04.017379 23020 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.125:42223 every 8 connection(s)
I20260806 03:52:04.017680 22779 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
I20260806 03:52:04.018770 23023 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:52:04.019310 23023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Bootstrap starting.
I20260806 03:52:04.022847 23023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:52:04.023216 23023 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: Bootstrap complete.
I20260806 03:52:04.023501 23023 raft_consensus.cc:348] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } }
I20260806 03:52:04.023671 23023 raft_consensus.cc:729] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 87612679f1764898a407f02fbd6a5c94, State: Initialized, Role: LEARNER
I20260806 03:52:04.024087 23023 consensus_queue.cc:260] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } }
I20260806 03:52:04.024376 23023 sys_catalog.cc:564] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:04.024400 23027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } }
I20260806 03:52:04.024585 23027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: This master's current role is: LEARNER
I20260806 03:52:04.026089 23042 catalog_manager.cc:1259] Loaded cluster ID: 2608214565024160a23a9ecdf1abeae3
I20260806 03:52:04.026607 23042 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: loading cluster ID for follower catalog manager: success
I20260806 03:52:04.027451 23042 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: acquiring CA information for follower catalog manager: success
I20260806 03:52:04.027714 23042 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:52:04.028396 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.019584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.12.15.125:42223)
0806 03:52:04.020900 (+ 1316us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.020902 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.020912 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:04.020936 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.020937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.020948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.021745 (+ 797us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.021777 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.022512 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.023331 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.023543 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.023595 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.024816 (+ 1221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.024817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.024817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.024825 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.027268 (+ 2443us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.027671 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.027673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.027673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.027739 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.027933 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.027934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.027934 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.028038 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.028119 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.028267 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.028337 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1262,"thread_start_us":95,"threads_started":1}
WARNING: no policy specified for oryx/127.12.15.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.124.keytab.
Entry for principal oryx/127.12.15.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.124.keytab.
I20260806 03:52:04.053269 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.124:39491
--webserver_interface=127.12.15.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.124.keytab
--principal=oryx/127.12.15.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:40955,127.12.15.125:42223,127.12.15.124:39491
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:04.135505 23045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:04.135710 23045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:04.135740 23045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:04.136870 23045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:04.136914 23045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:04.136933 23045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:04.136948 23045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:04.136965 23045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:04.138463 23045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:40955,127.12.15.125:42223,127.12.15.124:39491
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.124.keytab
--principal=oryx/127.12.15.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.124:39491
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.15.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:04.138708 23045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:04.138911 23045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:04.143482 23052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:04.143494 23054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:04.143709 23045 server_base.cc:1034] running on GCE node
W20260806 03:52:04.143821 23056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:04.144071 23045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:04.144353 23045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:04.145506 23045 hybrid_clock.cc:648] HybridClock initialized: now 1785988324145498 us; error 32 us; skew 500 ppm
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:04.147941 23045 init.cc:377] Logged in from keytab as oryx/127.12.15.124@KRBTEST.COM (short username oryx)
I20260806 03:52:04.148363 23045 webserver.cc:460] Webserver started at http://127.12.15.124:42281/ using document root <none> and password file <none>
I20260806 03:52:04.148576 23045 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:04.148627 23045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:04.148749 23045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:04.149757 23045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data/instance:
uuid: "2704f24a42474edea3016633c371c5f1"
format_stamp: "Formatted at 2026-08-06 03:52:04 on dist-test-slave-bqcl"
server_key: "2b2398f3e97c9ec619457cf471d17df5"
server_key_iv: "be388dbcd9578a9900252f4c39ba1a05"
server_key_version: "encryptionkey@0"
I20260806 03:52:04.150089 23045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal/instance:
uuid: "2704f24a42474edea3016633c371c5f1"
format_stamp: "Formatted at 2026-08-06 03:52:04 on dist-test-slave-bqcl"
server_key: "2b2398f3e97c9ec619457cf471d17df5"
server_key_iv: "be388dbcd9578a9900252f4c39ba1a05"
server_key_version: "encryptionkey@0"
I20260806 03:52:04.151315 23045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:52:04.151965 23063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.152107 23045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:04.152190 23045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
uuid: "2704f24a42474edea3016633c371c5f1"
format_stamp: "Formatted at 2026-08-06 03:52:04 on dist-test-slave-bqcl"
server_key: "2b2398f3e97c9ec619457cf471d17df5"
server_key_iv: "be388dbcd9578a9900252f4c39ba1a05"
server_key_version: "encryptionkey@0"
I20260806 03:52:04.152251 23045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:04.163394 23045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:04.165005 23045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:04.165140 23045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:04.168959 23045 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.124:39491
I20260806 03:52:04.168992 23145 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.124:39491 every 8 connection(s)
I20260806 03:52:04.169441 23045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
I20260806 03:52:04.170321 23146 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:04.171891 23146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.124@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:04.178552 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 23045
I20260806 03:52:04.180658 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.172717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58266 (local address 127.12.15.126:40955)
0806 03:52:04.172774 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.172776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.172848 (+ 72us) server_negotiation.cc:407] Connection header received
0806 03:52:04.173296 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.173298 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.173309 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.173371 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.174050 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.174600 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.175590 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.175779 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.175831 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.177860 (+ 2029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.177861 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.177862 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.177870 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.179370 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.179746 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.179747 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.179748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.179805 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.180032 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.180032 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.180033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.180153 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.180209 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.180471 (+ 262us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.180565 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:52:04.180876 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.172562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58266)
0806 03:52:04.172735 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.172776 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.173207 (+ 431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.173429 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.173432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.173746 (+ 314us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.173978 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.173983 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.174674 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.174674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.175497 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.175500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.175855 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.175856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.175955 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.176425 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.176433 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.177755 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.179444 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.179449 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.179451 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.179677 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.179859 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.179859 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.179860 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.179975 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.180209 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.180216 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.180350 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.180440 (+ 90us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.180661 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":40,"thread_start_us":79,"threads_started":1}
I20260806 03:52:04.181401 23146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.124@KRBTEST.COM
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.124@KRBTEST.COM for oryx/127.12.15.125@KRBTEST.COM
I20260806 03:52:04.187045 23149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.179124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40394 (local address 127.12.15.124:39491)
0806 03:52:04.179304 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.179308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.179324 (+ 16us) server_negotiation.cc:407] Connection header received
0806 03:52:04.179341 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.179343 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.179353 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.179414 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.179985 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.180657 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.181696 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.182024 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.182083 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.184265 (+ 2182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.184272 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.184273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.184280 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.185751 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.186313 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.186316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.186316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.186372 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.186535 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.186536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.186536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.186673 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.186805 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.186875 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.186947 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":113,"threads_started":1}
I20260806 03:52:04.187718 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.181644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.0.0.1:37080)
0806 03:52:04.181684 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.181710 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.181744 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.181904 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.181905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.182024 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.182067 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.182069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.182839 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.182841 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.183338 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.183339 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.183679 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.183680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.183747 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.184088 (+ 341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.184092 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.185319 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.186713 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.186714 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.186714 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.186865 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.187107 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.187109 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.187109 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.187175 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.187428 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.187429 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.187470 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.187532 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.187620 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 03:52:04.187724 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.181748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37080 (local address 127.12.15.125:42223)
0806 03:52:04.181808 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.181810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.181817 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:04.181840 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.181841 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.181851 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.181896 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.182153 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.182751 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.183408 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.183615 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.183675 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.185401 (+ 1726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.185403 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.185403 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.185411 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.186635 (+ 1224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.186923 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.186924 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.186924 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.186975 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.187238 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.187239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.187240 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.187365 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.187429 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.187561 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.187639 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:52:04.188063 23146 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.124@KRBTEST.COM for oryx/127.12.15.124@KRBTEST.COM
I20260806 03:52:04.193274 23148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.188282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:39491 (local address 127.0.0.1:40410)
0806 03:52:04.188327 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.188338 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.188375 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.188473 (+ 98us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.188473 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.188571 (+ 98us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.188597 (+ 26us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.188598 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.189106 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.189107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.189566 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.189567 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.189846 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.189846 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.189881 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.190221 (+ 340us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.190226 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.191224 (+ 998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.192555 (+ 1331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.192555 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.192556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.192700 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.192833 (+ 133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.192833 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.192834 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.192884 (+ 50us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.193053 (+ 169us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.193053 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.193084 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.193129 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.193215 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 03:52:04.193351 23149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.188335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40410 (local address 127.12.15.124:39491)
0806 03:52:04.188369 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.188370 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.188378 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:04.188424 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.188424 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.188431 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.188468 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.188633 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.189051 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.189619 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.189799 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.189841 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.191292 (+ 1451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.191293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.191293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.191299 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.192500 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.192752 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.192752 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.192753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.192793 (+ 40us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.192937 (+ 144us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.192938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.192938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.193014 (+ 76us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.193053 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.193145 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.193261 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":54,"server-negotiator.queue_time_us":17}
I20260806 03:52:04.194577 23146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Bootstrap starting.
I20260806 03:52:04.195380 23146 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:04.195652 23146 log.cc:824] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:04.196329 23146 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: No bootstrap required, opened a new log
I20260806 03:52:04.197721 23146 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } }
I20260806 03:52:04.197839 23146 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:04.197870 23146 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2704f24a42474edea3016633c371c5f1, State: Initialized, Role: FOLLOWER
I20260806 03:52:04.197985 23146 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } }
I20260806 03:52:04.198290 23146 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:04.198278 23153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } } }
I20260806 03:52:04.198359 23153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:52:04.199892 23167 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:52:04.199982 23167 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:52:04.201964 23045 master_runner.cc:428] Detected that this master 2704f24a42474edea3016633c371c5f1 is joining an existing cluster
I20260806 03:52:04.202003 23045 master_runner.cc:432] Initiating AddMaster RPC to add 127.12.15.124:39491
I20260806 03:52:04.209400 23149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.203338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40418 (local address 127.12.15.124:39491)
0806 03:52:04.203372 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.203373 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.203443 (+ 70us) server_negotiation.cc:407] Connection header received
0806 03:52:04.203508 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.203509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.203516 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.203557 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.203926 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.204619 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.205715 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.205986 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.206023 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.207090 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.207091 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.207091 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.207096 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.208302 (+ 1206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.208655 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.208656 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.208656 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.208702 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.208878 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.208878 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.208879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.208971 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.209013 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.209171 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.209304 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":19}
I20260806 03:52:04.209416 23176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.203178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:39491 (local address 127.0.0.1:40418)
0806 03:52:04.203394 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.203402 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.203465 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.203570 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.203571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.203771 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.203845 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.203847 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.204704 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.204705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.205627 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.205630 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.206108 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.206108 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.206198 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.206573 (+ 375us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.206577 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.207003 (+ 426us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.208372 (+ 1369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.208373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.208373 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.208587 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.208751 (+ 164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.208752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.208752 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.208818 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.209020 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.209021 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.209076 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.209156 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.209263 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":42,"thread_start_us":51,"threads_started":1}
I20260806 03:52:04.229907 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.202973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37094 (local address 127.12.15.125:42223)
0806 03:52:04.203109 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.203110 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.203151 (+ 41us) server_negotiation.cc:407] Connection header received
0806 03:52:04.203225 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.203226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.203236 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.203285 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.204132 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.204812 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.205792 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.206080 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.206113 (+ 33us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.227473 (+ 21360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.227475 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.227475 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.227483 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.228796 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.229110 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.229111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.229111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.229171 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.229376 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.229381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.229382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.229505 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.229561 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.229728 (+ 167us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.229805 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116}
I20260806 03:52:04.229921 23174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.202942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.0.0.1:37094)
0806 03:52:04.203096 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.203107 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.203164 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.203830 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.203831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.203985 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.204044 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.204046 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.204894 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.204895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.205709 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.205712 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.206210 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.206211 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.206274 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.226745 (+ 20471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.226750 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.227328 (+ 578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.228877 (+ 1549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.228878 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.228878 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.229046 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.229240 (+ 194us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.229242 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.229242 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.229318 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.229589 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.229589 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.229633 (+ 44us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.229697 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.229806 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":1,"thread_start_us":104,"threads_started":1}
I20260806 03:52:04.230247 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.203344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.12.15.126:40955)
0806 03:52:04.203391 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.203393 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.203399 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:52:04.203419 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.203419 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.203428 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.203472 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.203831 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.204598 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.205760 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.206060 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.206097 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.227433 (+ 21336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.227434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.227435 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.227445 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.229086 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.229433 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.229435 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.229435 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.229494 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.229670 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.229671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.229672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.229806 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.229867 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.230041 (+ 174us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.230149 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:52:04.230557 23175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.202950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58268)
0806 03:52:04.203254 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.203263 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.203297 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.203510 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.203511 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.203699 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.203757 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.203759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.204681 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.204682 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.205659 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.205662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.206205 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.206206 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.206274 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.226820 (+ 20546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.226824 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.227327 (+ 503us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.229180 (+ 1853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.229181 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.229181 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.229361 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.229549 (+ 188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.229550 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.229550 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.229610 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.229895 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.229896 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.229937 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.230340 (+ 403us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.230447 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":38,"thread_start_us":92,"threads_started":1}
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988323, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.126@KRBTEST.COM for oryx/127.12.15.124@KRBTEST.COM
I20260806 03:52:04.238334 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 22137
I20260806 03:52:04.238435 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.232511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:39491 (local address 127.0.0.1:40422)
0806 03:52:04.232554 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.232564 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.232601 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.232724 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.232724 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.232865 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.232902 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.232903 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.233472 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.233473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.233980 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.233981 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.234358 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.234359 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.234426 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.234769 (+ 343us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.234773 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.236079 (+ 1306us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.237464 (+ 1385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.237465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.237465 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.237620 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.237786 (+ 166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.237787 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.237787 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.237843 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.238141 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.238142 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.238189 (+ 47us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.238260 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.238352 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 03:52:04.251983 23149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.232570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.12.15.124:39491)
0806 03:52:04.232611 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.232612 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.232619 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:04.232652 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.232653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.232662 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.232718 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.232964 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.233409 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.234046 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.234292 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.234341 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.236153 (+ 1812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.236155 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.236155 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.236161 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.237391 (+ 1230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.237678 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.237679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.237679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.237733 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.237909 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.237910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.237910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.238046 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.238147 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.251754 (+ 13607us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.251874 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:52:04.252517 22051 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.12.15.124:39491: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "2704f24a42474edea3016633c371c5f1"
member_type: NON_VOTER
last_known_addr {
host: "127.12.15.124"
port: 39491
}
attrs {
promote: true
}
}
dest_uuid: "2dcfdad11d234a31a0a5a49e42fbda99"
cas_config_opid_index: 5
I20260806 03:52:04.252777 22051 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } }
I20260806 03:52:04.253206 22921 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2704f24a42474edea3016633c371c5f1 (127.12.15.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.254215 22921 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.12.15.124:39491
I20260806 03:52:04.254282 22921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.254340 22921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:04.255235 23107 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:04.256055 23107 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 FOLLOWER]: Refusing update from remote peer 2dcfdad11d234a31a0a5a49e42fbda99: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 03:52:04.257510 22919 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:04.258064 23045 master.cc:537] Master@127.12.15.124:39491 shutting down...
I20260806 03:52:04.259419 23107 raft_consensus.cc:858] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } }
W20260806 03:52:04.259505 23107 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } }
I20260806 03:52:04.260088 23107 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 87612679f1764898a407f02fbd6a5c94 (127.12.15.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } }
I20260806 03:52:04.261127 22919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.261224 22919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
W20260806 03:52:04.261817 22026 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 -> Peer 2704f24a42474edea3016633c371c5f1 (127.12.15.124:39491): Couldn't send request to peer 2704f24a42474edea3016633c371c5f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:52:04.262238 23153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dcfdad11d234a31a0a5a49e42fbda99. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } }
I20260806 03:52:04.262316 23153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 03:52:04.262347 23153 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } }
I20260806 03:52:04.262377 23153 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 03:52:04.262501 23189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } }
I20260806 03:52:04.262603 23189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260806 03:52:04.263559 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:43879
--local_ip_for_outbound_sockets=127.12.15.65
--tserver_master_addrs=127.12.15.126:40955
--webserver_port=36689
--webserver_interface=127.12.15.65
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab
--principal=oryx/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223,127.12.15.124:39491 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:52:04.265275 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.257531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.0.0.1:37098)
0806 03:52:04.261244 (+ 3713us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.261255 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.261306 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.261523 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.261524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.261526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:04.261573 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.261574 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.262904 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.262906 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.264106 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.264111 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.265014 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.265015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.265089 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.265106 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.265155 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.265196 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3694}
I20260806 03:52:04.265578 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.257644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37098 (local address 127.12.15.125:42223)
0806 03:52:04.261206 (+ 3562us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.261209 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.261356 (+ 147us) server_negotiation.cc:407] Connection header received
0806 03:52:04.261384 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.261385 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.261411 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.261464 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:04.261647 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.262543 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.264211 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.264839 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.264905 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.264982 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.265466 (+ 484us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.265512 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3509}
I20260806 03:52:04.266012 22985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 LEARNER]: Refusing update from remote peer 2dcfdad11d234a31a0a5a49e42fbda99: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260806 03:52:04.266235 22921 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260806 03:52:04.266760 22985 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 2704f24a42474edea3016633c371c5f1 (127.12.15.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.267242 23180 mvcc.cc:205] Tried to move back new op lower bound from 7315408172204216320 to 7315408172127469568. Current Snapshot: MvccSnapshot[applied={T|T < 7315408172204216320 or (T in {7315408172204216320})}]
I20260806 03:52:04.267866 23027 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dcfdad11d234a31a0a5a49e42fbda99. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.267952 23027 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: This master's current role is: LEARNER
I20260806 03:52:04.267927 23194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: NON_VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: true } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.268014 23194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: This master's current role is: LEARNER
I20260806 03:52:04.268293 22921 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: attempting to promote NON_VOTER 87612679f1764898a407f02fbd6a5c94 to VOTER
I20260806 03:52:04.268296 23045 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260806 03:52:04.268389 23045 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260806 03:52:04.268415 23045 tablet_replica.cc:330] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: stopping tablet replica
I20260806 03:52:04.268463 22921 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } }
I20260806 03:52:04.268977 22985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 LEARNER]: Refusing update from remote peer 2dcfdad11d234a31a0a5a49e42fbda99: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
W20260806 03:52:04.269021 22026 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 -> Peer 2704f24a42474edea3016633c371c5f1 (127.12.15.124:39491): Couldn't send request to peer 2704f24a42474edea3016633c371c5f1. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260806 03:52:04.269200 22920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260806 03:52:04.270267 22921 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 87612679f1764898a407f02fbd6a5c94 (127.12.15.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.271144 22921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.271215 22921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:04.271267 22921 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.271306 22921 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:04.271471 22985 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 87612679f1764898a407f02fbd6a5c94 (127.12.15.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.272487 23194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.272619 23194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:52:04.284114 23045 master.cc:552] Master@127.12.15.124:39491 shutdown complete.
I20260806 03:52:04.284165 23045 master_runner.cc:305] Clearing existing system tablet
I20260806 03:52:04.284287 23045 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:04.284333 23045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:04.285025 23045 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.285554 23197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.285650 23045 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:04.285699 23045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.285719 23045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
uuid: "2704f24a42474edea3016633c371c5f1"
format_stamp: "Formatted at 2026-08-06 03:52:04 on dist-test-slave-bqcl"
server_key: "2b2398f3e97c9ec619457cf471d17df5"
server_key_iv: "be388dbcd9578a9900252f4c39ba1a05"
server_key_version: "encryptionkey@0"
I20260806 03:52:04.286492 23045 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:04.287788 23045 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260806 03:52:04.287855 23045 log.cc:1196] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260806 03:52:04.288050 23045 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Deleting consensus metadata
I20260806 03:52:04.288166 23045 master_runner.cc:315] Copying system tablet from 127.12.15.126:40955
I20260806 03:52:04.288753 23045 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: tablet copy: Beginning tablet copy session from remote peer at address 127.12.15.126:40955
I20260806 03:52:04.296236 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.289100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58278 (local address 127.12.15.126:40955)
0806 03:52:04.289145 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.289147 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.289247 (+ 100us) server_negotiation.cc:407] Connection header received
0806 03:52:04.289311 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.289312 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.289322 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.289388 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.289746 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.290522 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.291549 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.291842 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.291908 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.293127 (+ 1219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.293129 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.293129 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.293137 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.294893 (+ 1756us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.295311 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.295312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.295312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.295405 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.295705 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.295706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.295706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.295836 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.295900 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.296063 (+ 163us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.296143 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:52:04.296259 23208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.289037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58278)
0806 03:52:04.289175 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.289186 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.289249 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.289393 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.289394 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.289599 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.289664 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.289667 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.290619 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.290620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.291461 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.291465 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.291969 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.291970 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.292042 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.292561 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.292567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.292976 (+ 409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.294981 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.294982 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.294983 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.295223 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.295475 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.295476 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.295477 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.295632 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.295900 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.295910 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.295960 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.296036 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.296157 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":70,"threads_started":1}
I20260806 03:52:04.296765 22081 tablet_copy_service.cc:140] P 2dcfdad11d234a31a0a5a49e42fbda99: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 2704f24a42474edea3016633c371c5f1 ({username='oryx', principal='oryx/127.12.15.124@KRBTEST.COM'} at 127.0.0.1:58278)
I20260806 03:52:04.296814 22081 tablet_copy_service.cc:161] P 2dcfdad11d234a31a0a5a49e42fbda99: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 2704f24a42474edea3016633c371c5f1 at {username='oryx', principal='oryx/127.12.15.124@KRBTEST.COM'} at 127.0.0.1:58278: session id = 2704f24a42474edea3016633c371c5f1-00000000000000000000000000000000
I20260806 03:52:04.297353 22081 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: Tablet Copy: opened 0 blocks and 1 log segments
I20260806 03:52:04.297817 23045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260806 03:52:04.299077 23045 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: tablet copy: Starting download of 0 data blocks...
I20260806 03:52:04.299258 23045 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: tablet copy: Starting download of 1 WAL segments...
I20260806 03:52:04.300032 23045 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260806 03:52:04.300846 22081 tablet_copy_service.cc:342] P 2dcfdad11d234a31a0a5a49e42fbda99: Request end of tablet copy session 2704f24a42474edea3016633c371c5f1-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.12.15.124@KRBTEST.COM'} at 127.0.0.1:58278
I20260806 03:52:04.300904 22081 tablet_copy_service.cc:433] P 2dcfdad11d234a31a0a5a49e42fbda99: ending tablet copy session 2704f24a42474edea3016633c371c5f1-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 2704f24a42474edea3016633c371c5f1
I20260806 03:52:04.303426 23045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:04.304677 23212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:04.304699 23045 server_base.cc:1034] running on GCE node
W20260806 03:52:04.304870 23209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:04.305003 23210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:04.305259 23045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:04.305406 23045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:04.306535 23045 hybrid_clock.cc:648] HybridClock initialized: now 1785988324306526 us; error 23 us; skew 500 ppm
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:04.308027 23045 init.cc:377] Logged in from keytab as oryx/127.12.15.124@KRBTEST.COM (short username oryx)
I20260806 03:52:04.308321 23045 webserver.cc:460] Webserver started at http://127.12.15.124:37235/ using document root <none> and password file <none>
I20260806 03:52:04.308432 23045 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:04.308465 23045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:04.314400 23045 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260806 03:52:04.314975 23221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.315119 23045 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:04.315200 23045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
uuid: "2704f24a42474edea3016633c371c5f1"
format_stamp: "Formatted at 2026-08-06 03:52:04 on dist-test-slave-bqcl"
server_key: "2b2398f3e97c9ec619457cf471d17df5"
server_key_iv: "be388dbcd9578a9900252f4c39ba1a05"
server_key_version: "encryptionkey@0"
I20260806 03:52:04.315258 23045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:04.324678 23045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:04.326608 23045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:04.329767 23045 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.124:39491
I20260806 03:52:04.329800 23292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.124:39491 every 8 connection(s)
I20260806 03:52:04.330193 23045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
I20260806 03:52:04.331146 23294 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:52:04.331663 23294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Bootstrap starting.
I20260806 03:52:04.334020 23294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:52:04.334263 23294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Bootstrap complete.
I20260806 03:52:04.334525 23294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } }
I20260806 03:52:04.334620 23294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 2704f24a42474edea3016633c371c5f1, State: Initialized, Role: LEARNER
I20260806 03:52:04.334697 23294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } }
I20260806 03:52:04.334893 23294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:04.334911 23300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.335098 23300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: LEARNER
I20260806 03:52:04.336344 23319 catalog_manager.cc:1259] Loaded cluster ID: 2608214565024160a23a9ecdf1abeae3
I20260806 03:52:04.336426 23319 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: loading cluster ID for follower catalog manager: success
I20260806 03:52:04.337591 23319 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: acquiring CA information for follower catalog manager: success
I20260806 03:52:04.337883 23319 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260806 03:52:04.360483 23190 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:04.360674 23190 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:04.360697 23190 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:04.361845 23190 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:04.361887 23190 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:04.361915 23190 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:04.363381 23190 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab
--principal=oryx/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:43879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=36689
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223,127.12.15.124:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23190
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:04.363610 23190 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:04.363821 23190 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:52:04.368593 23190 server_base.cc:1034] running on GCE node
W20260806 03:52:04.368523 23336 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:04.368567 23331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:04.368544 23334 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:04.368973 23190 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:04.369204 23190 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:04.370338 23190 hybrid_clock.cc:648] HybridClock initialized: now 1785988324370326 us; error 26 us; skew 500 ppm
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:04.372821 23190 init.cc:377] Logged in from keytab as oryx/127.12.15.65@KRBTEST.COM (short username oryx)
I20260806 03:52:04.373302 23190 webserver.cc:460] Webserver started at http://127.12.15.65:36689/ using document root <none> and password file <none>
I20260806 03:52:04.373505 23190 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:04.373569 23190 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:04.374913 23190 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:04.375459 23346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.375622 23190 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:04.375725 23190 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "34bcb6df749444de99b1fe01c02d6c6a"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "22bb46d529e37aa37fef1f1373bfa9d1"
server_key_iv: "894c404ed16da85b02abb9751a6cb708"
server_key_version: "encryptionkey@0"
I20260806 03:52:04.376096 23190 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:04.392362 23190 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:04.394315 23190 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:04.394425 23190 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:04.394606 23190 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:04.394897 23190 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:04.394930 23190 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.394953 23190 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:04.394975 23190 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.400561 23190 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:43879
I20260806 03:52:04.400605 23500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:43879 every 8 connection(s)
I20260806 03:52:04.400915 23190 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:52:04.401909 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 23190
I20260806 03:52:04.402058 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 22339
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.125@KRBTEST.COM
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.124@KRBTEST.COM
I20260806 03:52:04.413437 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.402004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:50335 (local address 127.12.15.126:40955)
0806 03:52:04.402046 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.402048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.402535 (+ 487us) server_negotiation.cc:407] Connection header received
0806 03:52:04.403213 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.403215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.403227 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.403255 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.404175 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.404934 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.406583 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.406878 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.406919 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.410686 (+ 3767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.410687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.410688 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.410695 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.412192 (+ 1497us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.412562 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.412563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.412563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.412624 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.412803 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.412804 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.412804 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.412932 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.412980 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.413271 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.413351 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:52:04.413460 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.401549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:33461 (local address 127.12.15.125:42223)
0806 03:52:04.401597 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.401599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.402202 (+ 603us) server_negotiation.cc:407] Connection header received
0806 03:52:04.403096 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.403097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.403112 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.403143 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.404164 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.404939 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.406582 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.406882 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.406919 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.410074 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.410075 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.410076 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.410085 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.411605 (+ 1520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.412354 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.412356 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.412356 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.412417 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.412720 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.412721 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.412722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.412871 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.412910 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.413285 (+ 375us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.413365 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:52:04.414037 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.401555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.12.15.65:33461)
0806 03:52:04.402141 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.402151 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.402984 (+ 833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.403306 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.403310 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.403769 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.404054 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.404060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.405072 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.405073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.406480 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.406483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.407013 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.407013 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.407153 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.407868 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.407872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.409966 (+ 2094us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.411700 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.411708 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.411710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.411982 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.412489 (+ 507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.412490 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.412491 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.412626 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.412957 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.412964 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.413122 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.413590 (+ 468us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.413811 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"mutex_wait_us":43,"thread_start_us":83,"threads_started":1}
I20260806 03:52:04.414048 23508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.401926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:50335)
0806 03:52:04.402259 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.402267 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.403108 (+ 841us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.403312 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.403313 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.403769 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.404054 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.404060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.405072 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.405073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.406480 (+ 1407us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.406483 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.407011 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.407012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.407153 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.407780 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.407788 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.410589 (+ 2801us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.412277 (+ 1688us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.412278 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.412278 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.412480 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.412686 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.412687 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.412687 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.412742 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.413015 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.413015 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.413122 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.413855 (+ 733us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.413954 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":76,"thread_start_us":69,"threads_started":1}
I20260806 03:52:04.414815 23505 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40955
I20260806 03:52:04.414913 23505 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.414937 23506 heartbeater.cc:347] Connected to a master server at 127.12.15.125:42223
I20260806 03:52:04.414918 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.401555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:39491 (local address 127.12.15.65:36489)
0806 03:52:04.401732 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.401773 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.402974 (+ 1201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.404626 (+ 1652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.404627 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.404817 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.404891 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.404893 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.406406 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.406407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.407961 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.407965 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.408652 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.408653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.408738 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.409173 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.409179 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.410961 (+ 1782us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.413693 (+ 2732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.413694 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.413695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.413903 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.414109 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.414110 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.414110 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.414201 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.414480 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.414481 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.414520 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.414572 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.414660 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"spinlock_wait_cycles":384,"thread_start_us":89,"threads_started":1}
I20260806 03:52:04.414999 23506 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.415117 23506 heartbeater.cc:511] Master 127.12.15.125:42223 requested a full tablet report, sending...
I20260806 03:52:04.415119 23505 heartbeater.cc:511] Master 127.12.15.126:40955 requested a full tablet report, sending...
I20260806 03:52:04.415597 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.401612 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:36489 (local address 127.12.15.124:39491)
0806 03:52:04.402271 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.402273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.402283 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:04.404467 (+ 2184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.404469 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.404485 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.404565 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.405005 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.406299 (+ 1294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.408071 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.408509 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.408584 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.412063 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.412064 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.412065 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.412072 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.413611 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.413980 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.413981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.413981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.414041 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.414269 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.414270 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.414270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.414399 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.414453 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.415424 (+ 971us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.415513 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":87,"threads_started":1}
I20260806 03:52:04.415588 22051 ts_manager.cc:194] Re-registered known tserver with Master: 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65:43879)
I20260806 03:52:04.415618 22974 ts_manager.cc:194] Registered new tserver with Master: 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65:43879)
I20260806 03:52:04.416031 22051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:50335
I20260806 03:52:04.417426 23502 heartbeater.cc:347] Connected to a master server at 127.12.15.124:39491
I20260806 03:52:04.417476 23502 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.417539 23502 heartbeater.cc:511] Master 127.12.15.124:39491 requested a full tablet report, sending...
I20260806 03:52:04.417536 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:40675
--local_ip_for_outbound_sockets=127.12.15.66
--tserver_master_addrs=127.12.15.126:40955
--webserver_port=44187
--webserver_interface=127.12.15.66
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab
--principal=oryx/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223,127.12.15.124:39491 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:52:04.417903 23244 ts_manager.cc:194] Registered new tserver with Master: 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65:43879)
W20260806 03:52:04.498728 23514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:04.498920 23514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:04.498942 23514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:04.500039 23514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:04.500079 23514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:04.500108 23514 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:04.501544 23514 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab
--principal=oryx/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:40675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=44187
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223,127.12.15.124:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:04.501756 23514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:04.501982 23514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:52:04.506826 23514 server_base.cc:1034] running on GCE node
W20260806 03:52:04.506799 23522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:04.506791 23524 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:04.506798 23529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:04.507261 23514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:04.507493 23514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:04.508627 23514 hybrid_clock.cc:648] HybridClock initialized: now 1785988324508614 us; error 32 us; skew 500 ppm
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:04.511062 23514 init.cc:377] Logged in from keytab as oryx/127.12.15.66@KRBTEST.COM (short username oryx)
I20260806 03:52:04.511498 23514 webserver.cc:460] Webserver started at http://127.12.15.66:44187/ using document root <none> and password file <none>
I20260806 03:52:04.511716 23514 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:04.511780 23514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:04.513036 23514 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:04.513574 23536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.513728 23514 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:04.513808 23514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "c7af4122ee524f78a548e4f2be47996a"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "1edc660724e20506c9f17c76004bb75c"
server_key_iv: "c1fb7fb4c50bab19675d3e8367313f55"
server_key_version: "encryptionkey@0"
I20260806 03:52:04.514118 23514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:04.542724 23514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:04.544929 23514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:04.545073 23514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:04.545331 23514 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:04.545636 23514 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:04.545671 23514 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.545701 23514 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:04.545718 23514 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.551365 23514 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:40675
I20260806 03:52:04.551414 23696 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:40675 every 8 connection(s)
I20260806 03:52:04.551744 23514 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:52:04.552412 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 23514
I20260806 03:52:04.552570 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 22555
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.124@KRBTEST.COM
I20260806 03:52:04.576352 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.559574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:36797 (local address 127.12.15.124:39491)
0806 03:52:04.564318 (+ 4744us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.564320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.565817 (+ 1497us) server_negotiation.cc:407] Connection header received
0806 03:52:04.566973 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.566974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.566989 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.567014 (+ 25us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.567910 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.568523 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.569766 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.570002 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.570036 (+ 34us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.573234 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.573235 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.573235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.573243 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.574812 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.575341 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.575342 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.575343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.575402 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.575654 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.575655 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.575655 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.575787 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.575826 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.576172 (+ 346us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.576259 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4713}
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:04.577215 23704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.552460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:39491 (local address 127.12.15.66:36797)
0806 03:52:04.565642 (+ 13182us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.565719 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.566855 (+ 1136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.567198 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.567202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.567550 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.567815 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.567820 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.568617 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.568618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.569673 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.569675 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.570115 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.570115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.570256 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.571044 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.571054 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.573130 (+ 2076us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.574924 (+ 1794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.574932 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.574935 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.575255 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.575464 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.575464 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.575465 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.575590 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.575868 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.575875 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.576033 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.576475 (+ 442us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.576698 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13153,"thread_start_us":100,"threads_started":1}
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.125@KRBTEST.COM
I20260806 03:52:04.577916 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:45395
--local_ip_for_outbound_sockets=127.12.15.67
--tserver_master_addrs=127.12.15.126:40955
--webserver_port=43991
--webserver_interface=127.12.15.67
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab
--principal=oryx/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223,127.12.15.124:39491 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:52:04.579130 23697 heartbeater.cc:347] Connected to a master server at 127.12.15.124:39491
I20260806 03:52:04.579231 23697 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.579435 23697 heartbeater.cc:511] Master 127.12.15.124:39491 requested a full tablet report, sending...
I20260806 03:52:04.579898 23244 ts_manager.cc:194] Registered new tserver with Master: c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675)
I20260806 03:52:04.580484 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.552458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.12.15.66:59623)
0806 03:52:04.552626 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.572466 (+ 19840us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.572561 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.572758 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.572759 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.572963 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.573028 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.573030 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.574518 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.574519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.575348 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.575351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.576052 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.576052 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.576117 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.576479 (+ 362us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.576483 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.577841 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.579481 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.579482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.579482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.579702 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.579892 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.579892 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.579893 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.579945 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.580192 (+ 247us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.580193 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.580231 (+ 38us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.580291 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.580379 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":91,"threads_started":1}
I20260806 03:52:04.580505 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.552448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:59623 (local address 127.12.15.125:42223)
0806 03:52:04.552499 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.552501 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.572530 (+ 20029us) server_negotiation.cc:407] Connection header received
0806 03:52:04.572633 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.572641 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.572657 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.572711 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.573121 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.574427 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.575436 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.575927 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.575972 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.577939 (+ 1967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.577941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.577941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.577949 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.579389 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.579778 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.579780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.579780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.579833 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.580003 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.580004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.580004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.580120 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.580156 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.580331 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.580414 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:52:04.580765 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.564258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:37371 (local address 127.12.15.126:40955)
0806 03:52:04.564358 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.564360 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.572730 (+ 8370us) server_negotiation.cc:407] Connection header received
0806 03:52:04.572827 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.572829 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.572842 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.572873 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.573295 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.573994 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.575016 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.575325 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.575363 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.577959 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.577960 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.577961 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.577969 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.579588 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.579964 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.579965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.579966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.580028 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.580243 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.580243 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.580244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.580369 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.580413 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.580612 (+ 199us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.580685 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 03:52:04.581005 23700 heartbeater.cc:347] Connected to a master server at 127.12.15.125:42223
I20260806 03:52:04.581068 23700 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.581139 23700 heartbeater.cc:511] Master 127.12.15.125:42223 requested a full tablet report, sending...
I20260806 03:52:04.581336 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.552757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:37371)
0806 03:52:04.572672 (+ 19915us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.572681 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.572755 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.572932 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.572933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.573125 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.573208 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.573210 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.574089 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.574090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.574928 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.574930 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.575439 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.575440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.575498 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.575868 (+ 370us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.575872 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.577870 (+ 1998us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.579691 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.579692 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.579693 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.579881 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.580101 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.580102 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.580102 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.580173 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.580458 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.580459 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.580498 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.580869 (+ 371us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.580969 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19894,"thread_start_us":52,"threads_started":1}
I20260806 03:52:04.581411 22974 ts_manager.cc:194] Registered new tserver with Master: c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675)
I20260806 03:52:04.581686 23698 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40955
I20260806 03:52:04.581735 23698 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.581791 23698 heartbeater.cc:511] Master 127.12.15.126:40955 requested a full tablet report, sending...
I20260806 03:52:04.582018 22051 ts_manager.cc:194] Re-registered known tserver with Master: c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675)
I20260806 03:52:04.582465 22051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:37371
W20260806 03:52:04.660820 23706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:04.661016 23706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:04.661038 23706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:04.662109 23706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:04.662169 23706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:04.662194 23706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:04.663599 23706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab
--principal=oryx/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:45395
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=43991
--tserver_master_addrs=127.12.15.126:40955,127.12.15.125:42223,127.12.15.124:39491
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:04.663793 23706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:04.663977 23706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:04.668730 23713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:04.668761 23714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:04.668819 23706 server_base.cc:1034] running on GCE node
W20260806 03:52:04.668730 23716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:04.669167 23706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:04.669401 23706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:04.670532 23706 hybrid_clock.cc:648] HybridClock initialized: now 1785988324670521 us; error 24 us; skew 500 ppm
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:04.672851 23706 init.cc:377] Logged in from keytab as oryx/127.12.15.67@KRBTEST.COM (short username oryx)
I20260806 03:52:04.673285 23706 webserver.cc:460] Webserver started at http://127.12.15.67:43991/ using document root <none> and password file <none>
I20260806 03:52:04.673496 23706 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:04.673560 23706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:04.674968 23706 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:04.675539 23726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.675685 23706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:04.675774 23706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "7f113aabda2943449519b0a57873f4ce"
format_stamp: "Formatted at 2026-08-06 03:52:03 on dist-test-slave-bqcl"
server_key: "6f439daa3b2ad355485a8144e6f9ddf5"
server_key_iv: "dbd1367fba801aa3702af739cf86488b"
server_key_version: "encryptionkey@0"
I20260806 03:52:04.676079 23706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:04.710381 23706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:04.712177 23706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:04.712314 23706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:04.712535 23706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:04.712833 23706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:04.712865 23706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.712895 23706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:04.712917 23706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:04.718657 23706 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:45395
I20260806 03:52:04.718734 23887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:45395 every 8 connection(s)
I20260806 03:52:04.719036 23706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:52:04.725033 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 23706
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.124@KRBTEST.COM
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
Aug 06 03:52:04 dist-test-slave-bqcl krb5kdc[21970](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988324, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.125@KRBTEST.COM
I20260806 03:52:04.730677 22130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.720381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:48709 (local address 127.12.15.126:40955)
0806 03:52:04.720456 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.720458 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.720469 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:52:04.720767 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.720768 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.720781 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.720816 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.722551 (+ 1735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.723022 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.723795 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.724044 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.724105 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.727155 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.727156 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.727157 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.727165 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.729421 (+ 2256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.729745 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.729746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.729746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.729802 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.730084 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.730084 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.730085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.730246 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.730301 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.730507 (+ 206us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.730587 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 03:52:04.730726 23025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.719744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:35409 (local address 127.12.15.125:42223)
0806 03:52:04.719816 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.719818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.720082 (+ 264us) server_negotiation.cc:407] Connection header received
0806 03:52:04.720766 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.720767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.720781 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.720816 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.721695 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.722459 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.724287 (+ 1828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.724553 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.724610 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.727665 (+ 3055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.727667 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.727667 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.727674 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.729169 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.729522 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.729523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.729523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.729592 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.730215 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.730215 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.730216 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.730337 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.730390 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.730570 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.730635 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 03:52:04.730790 23895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.719726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:39491 (local address 127.12.15.67:50037)
0806 03:52:04.720101 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.720111 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.720645 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.721135 (+ 490us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.721136 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.721339 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.721570 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.721575 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.722603 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.722604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.723843 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.723844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.724256 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.724257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.724357 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.724940 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.724945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.727094 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.728990 (+ 1896us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.728998 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.729000 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.729310 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.729596 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.729597 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.729597 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.729735 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.730064 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.730071 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.730250 (+ 179us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.730354 (+ 104us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.730552 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":273,"thread_start_us":97,"threads_started":1}
I20260806 03:52:04.731218 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.719763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:50037 (local address 127.12.15.124:39491)
0806 03:52:04.719816 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.719818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.720156 (+ 338us) server_negotiation.cc:407] Connection header received
0806 03:52:04.720729 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.720730 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.720741 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.720790 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.721655 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.722459 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.723922 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.724192 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.724240 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.727181 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.727182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.727183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.727189 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.728904 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.729406 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.729407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.729408 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.729509 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.729817 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.729819 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.729819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.729967 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.730036 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.731033 (+ 997us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.731123 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:52:04.731343 23897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.720171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:48709)
0806 03:52:04.720295 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.720302 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.720642 (+ 340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.720903 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.720914 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.721380 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.721571 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.721577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.723097 (+ 1520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.723098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.723718 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.723720 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.724117 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.724118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.724229 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.724765 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.724775 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.727061 (+ 2286us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.729498 (+ 2437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.729498 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.729499 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.729677 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.729867 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.729868 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.729868 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.729929 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.730325 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.730326 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.730365 (+ 39us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.730424 (+ 59us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.731237 (+ 813us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"mutex_wait_us":709,"thread_start_us":88,"threads_started":1}
I20260806 03:52:04.731424 23894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.719726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.12.15.67:35409)
0806 03:52:04.719918 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.719962 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.720632 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.720903 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.720908 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.721373 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:04.721571 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.721577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.722603 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.722604 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.723840 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.723843 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.724628 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.724629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.724710 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.726063 (+ 1353us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:04.726068 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:04.727577 (+ 1509us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:04.729261 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.729261 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.729262 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.729455 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.730038 (+ 583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:04.730039 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:04.730039 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:04.730102 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:04.730413 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:04.730413 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:04.730445 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.730501 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.731311 (+ 810us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"mutex_wait_us":808,"thread_start_us":89,"threads_started":1}
I20260806 03:52:04.731611 23889 heartbeater.cc:347] Connected to a master server at 127.12.15.124:39491
I20260806 03:52:04.731709 23889 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.731892 23889 heartbeater.cc:511] Master 127.12.15.124:39491 requested a full tablet report, sending...
I20260806 03:52:04.731978 23891 heartbeater.cc:347] Connected to a master server at 127.12.15.125:42223
I20260806 03:52:04.732028 23891 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.732092 23891 heartbeater.cc:511] Master 127.12.15.125:42223 requested a full tablet report, sending...
I20260806 03:52:04.732205 23890 heartbeater.cc:347] Connected to a master server at 127.12.15.126:40955
I20260806 03:52:04.732245 23890 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:04.732298 23890 heartbeater.cc:511] Master 127.12.15.126:40955 requested a full tablet report, sending...
I20260806 03:52:04.732370 23244 ts_manager.cc:194] Registered new tserver with Master: 7f113aabda2943449519b0a57873f4ce (127.12.15.67:45395)
I20260806 03:52:04.732407 22974 ts_manager.cc:194] Registered new tserver with Master: 7f113aabda2943449519b0a57873f4ce (127.12.15.67:45395)
I20260806 03:52:04.732980 22051 ts_manager.cc:194] Re-registered known tserver with Master: 7f113aabda2943449519b0a57873f4ce (127.12.15.67:45395)
I20260806 03:52:04.733417 22051 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:48709
I20260806 03:52:04.734398 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.729865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:39491 (local address 127.0.0.1:40436)
0806 03:52:04.729940 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.729949 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.729980 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.731473 (+ 1493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.731487 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.731490 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:04.731528 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.731530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.732302 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.732304 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.733343 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.733346 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.734176 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.734176 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.734236 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.734240 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.734285 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.734326 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61,"mutex_wait_us":1}
I20260806 03:52:04.734769 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.730008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40436 (local address 127.12.15.124:39491)
0806 03:52:04.731300 (+ 1292us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.731302 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.731310 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:04.731330 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.731331 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.731353 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.731423 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:04.731608 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.732213 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.733438 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.734026 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.734100 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.734642 (+ 542us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.734664 (+ 22us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.734700 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1278,"thread_start_us":44,"threads_started":1}
I20260806 03:52:04.735136 23259 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260806 03:52:04.735301 23300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2dcfdad11d234a31a0a5a49e42fbda99. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.735380 23300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: LEARNER
I20260806 03:52:04.739238 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.726473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.12.15.124:39491)
0806 03:52:04.726579 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.726581 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.730539 (+ 3958us) server_negotiation.cc:407] Connection header received
0806 03:52:04.731584 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.731585 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.731598 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.731637 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.733229 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.733883 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.734758 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.734973 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.735028 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.736499 (+ 1471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.736500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.736501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.736507 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.737998 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.738400 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.738401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.738401 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.738470 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.738710 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.738711 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.738711 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.738831 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.738928 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.739068 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.739141 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":62,"threads_started":1}
I20260806 03:52:04.739912 23904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.726218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58294 (local address 127.12.15.126:40955)
0806 03:52:04.727699 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.727701 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.727709 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:04.727732 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.727733 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.727746 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.727818 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.730582 (+ 2764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.734145 (+ 3563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.735080 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.735372 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.735434 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.737079 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.737080 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.737081 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.737088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.738602 (+ 1514us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.739001 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.739002 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.739002 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.739069 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.739272 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.739273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.739273 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.739404 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.739499 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.739678 (+ 179us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.739756 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1445,"thread_start_us":82,"threads_started":1}
I20260806 03:52:04.739917 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.726690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37114 (local address 127.12.15.125:42223)
0806 03:52:04.727044 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.727046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.727953 (+ 907us) server_negotiation.cc:407] Connection header received
0806 03:52:04.728805 (+ 852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.728806 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.728820 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.728880 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:04.729610 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.730361 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.735114 (+ 4753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.735414 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.735472 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.737080 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:04.737081 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:04.737081 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:04.737088 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:04.738759 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.739113 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.739115 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.739115 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.739172 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:04.739372 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:04.739373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:04.739374 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:04.739491 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:04.739566 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.739750 (+ 184us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.739829 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":52,"threads_started":1}
I20260806 03:52:04.741425 22051 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58294:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:52:04.743052 22051 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:52:04.743729 22920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: NON_VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: true } } }
I20260806 03:52:04.743834 22920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:04.744644 23914 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: attempting to promote NON_VOTER 2704f24a42474edea3016633c371c5f1 to VOTER
I20260806 03:52:04.744797 23914 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } }
I20260806 03:52:04.745217 23259 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 LEARNER]: Refusing update from remote peer 2dcfdad11d234a31a0a5a49e42fbda99: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 03:52:04.745290 22985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Refusing update from remote peer 2dcfdad11d234a31a0a5a49e42fbda99: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260806 03:52:04.745398 22920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 03:52:04.745500 23914 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 7, Time since last communication: 0.000s
I20260806 03:52:04.746150 22920 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2704f24a42474edea3016633c371c5f1 (127.12.15.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:04.746265 23259 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2704f24a42474edea3016633c371c5f1 (127.12.15.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:04.746392 22985 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 2704f24a42474edea3016633c371c5f1 (127.12.15.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:04.747012 23300 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:04.747087 23300 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:52:04.747054 22920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:04.747134 22920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:04.747196 22920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:04.747244 22920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:04.747284 22920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:04.747270 23194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:04.747328 22920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:04.747345 23194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:52:04.752704 22916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.748999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:45395 (local address 127.0.0.1:34028)
0806 03:52:04.749029 (+ 30us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.749040 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.749082 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.749489 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.749490 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.749493 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:04.749536 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.749538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.750902 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.750903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.751760 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.751763 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.752449 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.752450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.752520 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.752524 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.752594 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.752632 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17,"mutex_wait_us":41}
I20260806 03:52:04.752735 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.749123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34028 (local address 127.12.15.67:45395)
0806 03:52:04.749314 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.749319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.749329 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:04.749364 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.749366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.749382 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.749607 (+ 225us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:04.749660 (+ 53us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.750804 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.751842 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.752367 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.752439 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.752515 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.752607 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.752644 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
I20260806 03:52:04.753207 23925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.749308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:43879 (local address 127.0.0.1:55284)
0806 03:52:04.749426 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.749435 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.749475 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.749929 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.749929 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.749931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:04.749984 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.749986 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.750898 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.750899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.752105 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.752107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.752964 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.752966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.753036 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.753041 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.753096 (+ 55us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.753138 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":89,"thread_start_us":82,"threads_started":1}
I20260806 03:52:04.753291 23922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.749116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:40675 (local address 127.0.0.1:46622)
0806 03:52:04.749256 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:04.749266 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:04.749315 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:04.749637 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:04.749638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:04.749640 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:04.749672 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.749673 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.750960 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.750961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.752101 (+ 1140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:04.752104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.753106 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.753107 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:04.753160 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.753163 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:04.753191 (+ 28us) client_negotiation.cc:240] Negotiation successful
0806 03:52:04.753224 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260806 03:52:04.753587 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.749231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46622 (local address 127.12.15.66:40675)
0806 03:52:04.749405 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.749410 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.749419 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:04.749456 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.749458 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.749472 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.749740 (+ 268us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:04.749802 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.750868 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.752199 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.753000 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.753351 (+ 351us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.753454 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.753479 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.753522 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":77,"threads_started":1}
I20260806 03:52:04.753698 23795 tablet_service.cc:1467] Processing CreateTablet for tablet 2c94ceabd461426cb1757e18b91e78f0 (DEFAULT_TABLE table=test-table [id=fab922d721dd4c82810f4e3acc14a83f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:04.754058 23795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c94ceabd461426cb1757e18b91e78f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:04.754257 23602 tablet_service.cc:1467] Processing CreateTablet for tablet 2c94ceabd461426cb1757e18b91e78f0 (DEFAULT_TABLE table=test-table [id=fab922d721dd4c82810f4e3acc14a83f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:04.754566 23602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c94ceabd461426cb1757e18b91e78f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:04.757123 23929 tablet_bootstrap.cc:492] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Bootstrap starting.
I20260806 03:52:04.753237 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:04.749552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55284 (local address 127.12.15.65:43879)
0806 03:52:04.749766 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:04.749770 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:04.749781 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:52:04.749822 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:04.749824 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:04.749841 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:04.749919 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:04.750048 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.750808 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.752214 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:04.752842 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:04.752925 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:04.753020 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:04.753110 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 03:52:04.753156 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":93,"threads_started":1}
I20260806 03:52:04.757432 23930 tablet_bootstrap.cc:492] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Bootstrap starting.
I20260806 03:52:04.757891 23929 tablet_bootstrap.cc:654] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:04.758054 23930 tablet_bootstrap.cc:654] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:04.757902 23409 tablet_service.cc:1467] Processing CreateTablet for tablet 2c94ceabd461426cb1757e18b91e78f0 (DEFAULT_TABLE table=test-table [id=fab922d721dd4c82810f4e3acc14a83f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:04.758311 23929 log.cc:824] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:04.758333 23409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2c94ceabd461426cb1757e18b91e78f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:04.758404 23930 log.cc:824] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:04.759064 23929 tablet_bootstrap.cc:492] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: No bootstrap required, opened a new log
I20260806 03:52:04.759132 23929 ts_tablet_manager.cc:1397] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:04.759166 23930 tablet_bootstrap.cc:492] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: No bootstrap required, opened a new log
I20260806 03:52:04.759232 23930 ts_tablet_manager.cc:1397] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:52:04.761121 23929 raft_consensus.cc:348] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:04.761257 23929 raft_consensus.cc:374] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:04.761293 23929 raft_consensus.cc:729] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f113aabda2943449519b0a57873f4ce, State: Initialized, Role: FOLLOWER
I20260806 03:52:04.761409 23929 consensus_queue.cc:260] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:04.761674 23929 ts_tablet_manager.cc:1428] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:52:04.761701 23890 heartbeater.cc:503] Master 127.12.15.126:40955 was elected leader, sending a full tablet report...
I20260806 03:52:04.761672 23930 raft_consensus.cc:348] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:04.761816 23930 raft_consensus.cc:374] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:04.761847 23930 raft_consensus.cc:729] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7af4122ee524f78a548e4f2be47996a, State: Initialized, Role: FOLLOWER
I20260806 03:52:04.762168 23930 consensus_queue.cc:260] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:04.762427 23930 ts_tablet_manager.cc:1428] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 03:52:04.762552 23698 heartbeater.cc:503] Master 127.12.15.126:40955 was elected leader, sending a full tablet report...
I20260806 03:52:04.763077 23933 tablet_bootstrap.cc:492] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Bootstrap starting.
I20260806 03:52:04.763689 23933 tablet_bootstrap.cc:654] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:04.763988 23933 log.cc:824] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:04.764658 23933 tablet_bootstrap.cc:492] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: No bootstrap required, opened a new log
I20260806 03:52:04.764724 23933 ts_tablet_manager.cc:1397] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:04.766024 23933 raft_consensus.cc:348] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:04.766167 23933 raft_consensus.cc:374] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:04.766206 23933 raft_consensus.cc:729] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34bcb6df749444de99b1fe01c02d6c6a, State: Initialized, Role: FOLLOWER
I20260806 03:52:04.766284 23933 consensus_queue.cc:260] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:04.766459 23933 ts_tablet_manager.cc:1428] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:52:04.766530 23505 heartbeater.cc:503] Master 127.12.15.126:40955 was elected leader, sending a full tablet report...
W20260806 03:52:04.802498 23701 tablet.cc:2369] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:52:04.901753 23507 tablet.cc:2369] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:52:04.969785 23892 tablet.cc:2369] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:52:05.037258 23938 raft_consensus.cc:482] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:05.037380 23938 raft_consensus.cc:504] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:05.037708 23938 leader_election.cc:288] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f113aabda2943449519b0a57873f4ce (127.12.15.67:45395), c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675)
I20260806 03:52:05.040550 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.037954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:51649 (local address 127.12.15.67:45395)
0806 03:52:05.038005 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:05.038008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:05.038021 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:52:05.038182 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:05.038183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:05.038193 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:05.038244 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:05.038400 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.038921 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.039884 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.040242 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.040295 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.040304 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:05.040440 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 03:52:05.040477 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:52:05.040557 23924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.038109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:46139 (local address 127.12.15.66:40675)
0806 03:52:05.038197 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:05.038199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:05.038225 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:52:05.038307 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:05.038308 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:05.038318 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:05.038365 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:05.038490 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.038948 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.039885 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.040259 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.040324 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.040333 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:05.040456 (+ 123us) server_negotiation.cc:299] Negotiation successful
0806 03:52:05.040490 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 03:52:05.040563 23939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.037863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:45395 (local address 127.12.15.65:51649)
0806 03:52:05.037972 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:05.037984 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:05.038038 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:05.038275 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:05.038277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:05.038279 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:05.038339 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:05.038341 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.038991 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.038992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:05.039803 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:05.039805 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.040311 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.040312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:05.040377 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.040381 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:05.040423 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 03:52:05.040468 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91,"mutex_wait_us":50,"thread_start_us":55,"threads_started":1}
I20260806 03:52:05.040566 23942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.038026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:40675 (local address 127.12.15.65:46139)
0806 03:52:05.038158 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:05.038169 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:05.038241 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:05.038371 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:05.038372 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:05.038373 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:05.038435 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:05.038436 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.039025 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.039026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:05.039801 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:05.039803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.040337 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.040338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:05.040395 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.040398 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:05.040447 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:52:05.040485 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":41,"threads_started":1}
I20260806 03:52:05.040891 23631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c94ceabd461426cb1757e18b91e78f0" candidate_uuid: "34bcb6df749444de99b1fe01c02d6c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7af4122ee524f78a548e4f2be47996a" is_pre_election: true
I20260806 03:52:05.041011 23936 raft_consensus.cc:482] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:05.040977 23817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c94ceabd461426cb1757e18b91e78f0" candidate_uuid: "34bcb6df749444de99b1fe01c02d6c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f113aabda2943449519b0a57873f4ce" is_pre_election: true
I20260806 03:52:05.041083 23936 raft_consensus.cc:504] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:05.041108 23817 raft_consensus.cc:2454] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34bcb6df749444de99b1fe01c02d6c6a in term 0.
I20260806 03:52:05.041239 23631 raft_consensus.cc:2454] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 34bcb6df749444de99b1fe01c02d6c6a in term 0.
I20260806 03:52:05.041317 23348 leader_election.cc:302] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34bcb6df749444de99b1fe01c02d6c6a, 7f113aabda2943449519b0a57873f4ce; no voters:
I20260806 03:52:05.041356 23936 leader_election.cc:288] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f113aabda2943449519b0a57873f4ce (127.12.15.67:45395), 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65:43879)
I20260806 03:52:05.041435 23938 raft_consensus.cc:2781] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:52:05.041487 23938 raft_consensus.cc:482] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:05.041509 23938 raft_consensus.cc:3037] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:05.042347 23938 raft_consensus.cc:504] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:05.042443 23938 leader_election.cc:288] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [CANDIDATE]: Term 1 election: Requested vote from peers 7f113aabda2943449519b0a57873f4ce (127.12.15.67:45395), c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675)
I20260806 03:52:05.042642 23631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c94ceabd461426cb1757e18b91e78f0" candidate_uuid: "34bcb6df749444de99b1fe01c02d6c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7af4122ee524f78a548e4f2be47996a"
I20260806 03:52:05.042722 23631 raft_consensus.cc:3037] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:05.042773 23817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c94ceabd461426cb1757e18b91e78f0" candidate_uuid: "34bcb6df749444de99b1fe01c02d6c6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f113aabda2943449519b0a57873f4ce"
I20260806 03:52:05.042847 23817 raft_consensus.cc:3037] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:05.043607 23631 raft_consensus.cc:2454] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34bcb6df749444de99b1fe01c02d6c6a in term 1.
I20260806 03:52:05.043707 23817 raft_consensus.cc:2454] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 34bcb6df749444de99b1fe01c02d6c6a in term 1.
I20260806 03:52:05.043882 23348 leader_election.cc:302] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34bcb6df749444de99b1fe01c02d6c6a, 7f113aabda2943449519b0a57873f4ce; no voters:
I20260806 03:52:05.043991 23938 raft_consensus.cc:2781] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:05.044095 23938 raft_consensus.cc:686] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 1 LEADER]: Becoming Leader. State: Replica: 34bcb6df749444de99b1fe01c02d6c6a, State: Running, Role: LEADER
I20260806 03:52:05.044178 23938 consensus_queue.cc:237] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:05.044818 23705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.041504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:45395 (local address 127.12.15.66:47077)
0806 03:52:05.041545 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:05.041554 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:05.041589 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:05.041856 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:05.041857 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:05.041858 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:05.041912 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:05.041913 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.042745 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.042746 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:05.043855 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:05.043858 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.044586 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.044587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:05.044666 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.044671 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:52:05.044715 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:52:05.044753 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23,"mutex_wait_us":38}
I20260806 03:52:05.044994 23923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.041561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:47077 (local address 127.12.15.67:45395)
0806 03:52:05.041768 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:05.041769 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:05.041775 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:52:05.041793 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:05.041794 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:05.041802 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:05.041855 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:05.041968 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.042672 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.043921 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.044454 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.044518 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.044526 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:05.044909 (+ 383us) server_negotiation.cc:299] Negotiation successful
0806 03:52:05.044944 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191}
I20260806 03:52:05.044916 22051 catalog_manager.cc:5495] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a reported cstate change: term changed from 0 to 1, leader changed from <none> to 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65). New cstate: current_term: 1 leader_uuid: "34bcb6df749444de99b1fe01c02d6c6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } health_report { overall_health: HEALTHY } } }
I20260806 03:52:05.045356 23817 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c94ceabd461426cb1757e18b91e78f0" candidate_uuid: "c7af4122ee524f78a548e4f2be47996a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f113aabda2943449519b0a57873f4ce" is_pre_election: true
I20260806 03:52:05.045441 23817 raft_consensus.cc:2379] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate c7af4122ee524f78a548e4f2be47996a in current term 1: Already voted for candidate 34bcb6df749444de99b1fe01c02d6c6a in this term.
I20260806 03:52:05.045998 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.041671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:51615 (local address 127.12.15.65:43879)
0806 03:52:05.041730 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:05.041732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:05.041739 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:05.041807 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:05.041808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:05.041815 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:05.041860 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:05.042027 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.042712 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.043953 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.044464 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.044518 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.044526 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:05.045912 (+ 1386us) server_negotiation.cc:299] Negotiation successful
0806 03:52:05.045946 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:52:05.046010 23702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.041605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:43879 (local address 127.12.15.66:51615)
0806 03:52:05.041685 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:05.041705 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:05.041745 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:05.041899 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:05.041900 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:05.041902 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:05.041958 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:05.041960 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.042790 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.042791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:05.043864 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:05.043867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.045759 (+ 1892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.045760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:05.045842 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.045846 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:05.045892 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:52:05.045928 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 03:52:05.046576 23433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2c94ceabd461426cb1757e18b91e78f0" candidate_uuid: "c7af4122ee524f78a548e4f2be47996a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bcb6df749444de99b1fe01c02d6c6a" is_pre_election: true
I20260806 03:52:05.046851 23540 leader_election.cc:302] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: c7af4122ee524f78a548e4f2be47996a; no voters: 34bcb6df749444de99b1fe01c02d6c6a, 7f113aabda2943449519b0a57873f4ce
I20260806 03:52:05.046985 23936 raft_consensus.cc:2726] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 03:52:05.103009 23926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.100685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55292 (local address 127.12.15.65:43879)
0806 03:52:05.100752 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:05.100754 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:05.100763 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:05.100802 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:05.100803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:05.100824 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:05.100884 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:05.101027 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.101501 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.102260 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.102457 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.102515 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.102676 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:05.102802 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:05.102876 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:05.102909 (+ 33us) server_negotiation.cc:299] Negotiation successful
0806 03:52:05.102928 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:52:05.104772 23817 raft_consensus.cc:1261] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 1 FOLLOWER]: Refusing update from remote peer 34bcb6df749444de99b1fe01c02d6c6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:05.104831 23631 raft_consensus.cc:1261] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Refusing update from remote peer 34bcb6df749444de99b1fe01c02d6c6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:05.105167 23938 consensus_queue.cc:1035] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:05.105320 23943 consensus_queue.cc:1035] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:05.108306 23950 mvcc.cc:205] Tried to move back new op lower bound from 7315408179626577920 to 7315408179381489664. Current Snapshot: MvccSnapshot[applied={T|T < 7315408179626577920}]
I20260806 03:52:05.110078 22049 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:05.110198 22049 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58294:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:05.111479 22049 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2dcfdad11d234a31a0a5a49e42fbda99: Sending DeleteTablet for 3 replicas of tablet 2c94ceabd461426cb1757e18b91e78f0
I20260806 03:52:05.111889 23795 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c94ceabd461426cb1757e18b91e78f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:05 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:34028
I20260806 03:52:05.111974 23602 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c94ceabd461426cb1757e18b91e78f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:05 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:46622
I20260806 03:52:05.111965 23409 tablet_service.cc:1514] Processing DeleteTablet for tablet 2c94ceabd461426cb1757e18b91e78f0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:05 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:55284
I20260806 03:52:05.112325 23963 tablet_replica.cc:330] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: stopping tablet replica
I20260806 03:52:05.112424 23963 raft_consensus.cc:2229] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:05.112529 23963 raft_consensus.cc:2258] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:05.112702 23961 tablet_replica.cc:330] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: stopping tablet replica
I20260806 03:52:05.112864 23961 raft_consensus.cc:2229] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:05.113015 23961 raft_consensus.cc:2258] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:05.113014 23963 ts_tablet_manager.cc:1905] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:05.113490 23961 ts_tablet_manager.cc:1905] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:05.113592 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 21990
I20260806 03:52:05.114665 23963 ts_tablet_manager.cc:1918] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:05.114768 23963 log.cc:1196] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/2c94ceabd461426cb1757e18b91e78f0
I20260806 03:52:05.115060 23963 ts_tablet_manager.cc:1939] T 2c94ceabd461426cb1757e18b91e78f0 P c7af4122ee524f78a548e4f2be47996a: Deleting consensus metadata
I20260806 03:52:05.123128 23961 ts_tablet_manager.cc:1918] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:05.123292 23961 log.cc:1196] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/2c94ceabd461426cb1757e18b91e78f0
I20260806 03:52:05.123627 23961 ts_tablet_manager.cc:1939] T 2c94ceabd461426cb1757e18b91e78f0 P 7f113aabda2943449519b0a57873f4ce: Deleting consensus metadata
I20260806 03:52:05.124764 23964 tablet_replica.cc:330] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: stopping tablet replica
I20260806 03:52:05.124969 23964 raft_consensus.cc:2229] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:52:05.125178 23964 raft_consensus.cc:2258] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:05.125830 23964 ts_tablet_manager.cc:1905] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:05.127436 23964 ts_tablet_manager.cc:1918] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:05.127573 23964 log.cc:1196] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/2c94ceabd461426cb1757e18b91e78f0
I20260806 03:52:05.127921 23964 ts_tablet_manager.cc:1939] T 2c94ceabd461426cb1757e18b91e78f0 P 34bcb6df749444de99b1fe01c02d6c6a: Deleting consensus metadata
W20260806 03:52:05.128309 23349 connection.cc:441] server connection from 127.0.0.1:55284 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55284 (request call id 1) could send its response
W20260806 03:52:05.128322 23540 connection.cc:535] server connection from 127.0.0.1:46622 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 03:52:05.128615 23732 connection.cc:535] server connection from 127.0.0.1:34028 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:52:05.131744 23907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.112833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40444 (local address 127.12.15.124:39491)
0806 03:52:05.112867 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:05.112869 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:05.122350 (+ 9481us) server_negotiation.cc:407] Connection header received
0806 03:52:05.122457 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:05.122458 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:05.122472 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:05.122502 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:05.123295 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.125141 (+ 1846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.126216 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.126493 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.126541 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.128575 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:05.128576 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:05.128577 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:05.128584 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:05.130307 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:05.130672 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:05.130673 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:05.130673 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:05.130735 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:05.130938 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:05.130939 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:05.130939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:05.131074 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:05.131164 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:05.131576 (+ 412us) server_negotiation.cc:299] Negotiation successful
0806 03:52:05.131654 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:52:05.137468 23909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.130312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37130 (local address 127.12.15.125:42223)
0806 03:52:05.130355 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:05.130357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:05.130456 (+ 99us) server_negotiation.cc:407] Connection header received
0806 03:52:05.130559 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:05.130560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:05.130573 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:05.130601 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:05.131245 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.132002 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.133117 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.133348 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.133405 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.134800 (+ 1395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:05.134801 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:05.134802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:05.134809 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:05.136335 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:05.136664 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:05.136666 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:05.136666 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:05.136719 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:05.136913 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:05.136914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:05.136914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:05.137034 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:05.137105 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:05.137296 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 03:52:05.137374 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:52:05.137780 23510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:05.130682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.12.15.124:39491)
0806 03:52:05.130733 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:05.130735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:05.131009 (+ 274us) server_negotiation.cc:407] Connection header received
0806 03:52:05.131089 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:05.131090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:05.131099 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:05.131125 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:05.131738 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.132386 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.133158 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:05.133492 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:05.133559 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:05.135133 (+ 1574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:05.135143 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:05.135144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:05.135151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:05.136561 (+ 1410us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:05.136904 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:05.136905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:05.136905 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:05.136964 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:05.137145 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:05.137145 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:05.137146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:05.137267 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:05.137352 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:05.137613 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 03:52:05.137688 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 03:52:06.109642 23977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.109337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:56145)
0806 03:52:06.109469 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.109578 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":66,"threads_started":1}
W20260806 03:52:06.109923 23890 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:40955 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:06.110138 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.109855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:46243)
0806 03:52:06.109975 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.110069 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":55,"threads_started":1}
W20260806 03:52:06.110445 23505 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:40955 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:06.110730 23979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.110538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:47799)
0806 03:52:06.110633 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.110695 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79,"thread_start_us":53,"threads_started":1}
W20260806 03:52:06.110916 23698 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:40955 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:06.612236 23981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2dcfdad11d234a31a0a5a49e42fbda99)
I20260806 03:52:06.612249 23980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 2dcfdad11d234a31a0a5a49e42fbda99)
I20260806 03:52:06.612358 23980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } }
I20260806 03:52:06.612367 23981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } }
I20260806 03:52:06.612610 23980 leader_election.cc:288] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955), 2704f24a42474edea3016633c371c5f1 (127.12.15.124:39491)
I20260806 03:52:06.612775 23981 leader_election.cc:288] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955), 87612679f1764898a407f02fbd6a5c94 (127.12.15.125:42223)
I20260806 03:52:06.613029 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.612705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:57992)
0806 03:52:06.612937 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.612990 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":51,"threads_started":1}
I20260806 03:52:06.612908 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.612690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:57988)
0806 03:52:06.612807 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.612863 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":53,"threads_started":1}
W20260806 03:52:06.613186 23222 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
W20260806 03:52:06.613273 22948 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:06.613626 23984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.613541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58002)
0806 03:52:06.613576 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.613598 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260806 03:52:06.613756 23986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.613674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58008)
0806 03:52:06.613708 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.613730 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 03:52:06.613770 23222 leader_election.cc:334] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955): Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
W20260806 03:52:06.613940 22948 leader_election.cc:334] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955): Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:06.615717 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.612715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:42223 (local address 127.0.0.1:39308)
0806 03:52:06.612998 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.613020 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:06.613063 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:06.613281 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:06.613282 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:06.613284 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:06.613337 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:06.613339 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:06.614237 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:06.614238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:06.614956 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:06.614957 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:06.615521 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:06.615521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:06.615573 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:06.615576 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:06.615619 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:52:06.615651 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":36,"threads_started":1}
I20260806 03:52:06.615727 23987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.612827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39308 (local address 127.12.15.125:42223)
0806 03:52:06.613152 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:06.613153 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:06.613160 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:06.613178 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:06.613178 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:06.613187 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:06.613397 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:06.613433 (+ 36us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:06.614163 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:06.615029 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:06.615451 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:06.615514 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:06.615524 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:06.615632 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 03:52:06.615668 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":62,"threads_started":1}
I20260806 03:52:06.615795 23988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.612962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35986 (local address 127.12.15.124:39491)
0806 03:52:06.613074 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:06.613076 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:06.613085 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:06.613110 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:06.613111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:06.613121 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:06.613187 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:06.613383 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:06.614116 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:06.615119 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:06.615497 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:06.615557 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:06.615566 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:06.615685 (+ 119us) server_negotiation.cc:299] Negotiation successful
0806 03:52:06.615725 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":47,"threads_started":1}
I20260806 03:52:06.615936 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.612850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:39491 (local address 127.0.0.1:35986)
0806 03:52:06.612971 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.612983 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:06.613039 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:06.613244 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:06.613245 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:06.613248 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:06.613304 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:06.613306 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:06.614213 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:06.614214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:06.615024 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:06.615026 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:06.615575 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:06.615576 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:06.615624 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:06.615627 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:06.615673 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:52:06.615710 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":35,"threads_started":1}
I20260806 03:52:06.616158 22985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2704f24a42474edea3016633c371c5f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "87612679f1764898a407f02fbd6a5c94" is_pre_election: true
I20260806 03:52:06.616214 23259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87612679f1764898a407f02fbd6a5c94" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2704f24a42474edea3016633c371c5f1" is_pre_election: true
I20260806 03:52:06.616285 23259 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 87612679f1764898a407f02fbd6a5c94 in term 1.
I20260806 03:52:06.616279 22985 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2704f24a42474edea3016633c371c5f1 in term 1.
I20260806 03:52:06.616477 23223 leader_election.cc:302] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2704f24a42474edea3016633c371c5f1, 87612679f1764898a407f02fbd6a5c94; no voters: 2dcfdad11d234a31a0a5a49e42fbda99
I20260806 03:52:06.616483 22948 leader_election.cc:302] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2704f24a42474edea3016633c371c5f1, 87612679f1764898a407f02fbd6a5c94; no voters: 2dcfdad11d234a31a0a5a49e42fbda99
I20260806 03:52:06.616576 23980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:52:06.616614 23980 raft_consensus.cc:482] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2dcfdad11d234a31a0a5a49e42fbda99)
I20260806 03:52:06.616640 23980 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:52:06.617316 23980 raft_consensus.cc:504] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } }
I20260806 03:52:06.617410 23980 leader_election.cc:288] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [CANDIDATE]: Term 2 election: Requested vote from peers 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955), 2704f24a42474edea3016633c371c5f1 (127.12.15.124:39491)
I20260806 03:52:06.617480 23981 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260806 03:52:06.617523 23981 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 2dcfdad11d234a31a0a5a49e42fbda99)
I20260806 03:52:06.617550 23981 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:52:06.617620 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.617508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58018)
0806 03:52:06.617563 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.617595 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 03:52:06.617660 23259 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "87612679f1764898a407f02fbd6a5c94" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "2704f24a42474edea3016633c371c5f1"
I20260806 03:52:06.617916 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.617858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58026)
0806 03:52:06.617882 (+ 24us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.617903 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 03:52:06.617956 22948 leader_election.cc:334] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955): Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:06.618424 23981 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } }
I20260806 03:52:06.618552 23981 leader_election.cc:288] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 2 election: Requested vote from peers 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955), 87612679f1764898a407f02fbd6a5c94 (127.12.15.125:42223)
I20260806 03:52:06.618577 23259 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 87612679f1764898a407f02fbd6a5c94 in current term 2: Already voted for candidate 2704f24a42474edea3016633c371c5f1 in this term.
I20260806 03:52:06.618685 22985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2704f24a42474edea3016633c371c5f1" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "87612679f1764898a407f02fbd6a5c94"
I20260806 03:52:06.618767 22985 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 2704f24a42474edea3016633c371c5f1 in current term 2: Already voted for candidate 87612679f1764898a407f02fbd6a5c94 in this term.
I20260806 03:52:06.618791 22948 leader_election.cc:302] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 87612679f1764898a407f02fbd6a5c94; no voters: 2704f24a42474edea3016633c371c5f1, 2dcfdad11d234a31a0a5a49e42fbda99
I20260806 03:52:06.618852 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.618764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58032)
0806 03:52:06.618796 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.618825 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 03:52:06.618901 23980 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 03:52:06.619163 23982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:06.619098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58046)
0806 03:52:06.619121 (+ 23us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:06.619146 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":12}
W20260806 03:52:06.619230 23222 leader_election.cc:334] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955): Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:06.619302 23222 leader_election.cc:302] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2704f24a42474edea3016633c371c5f1; no voters: 2dcfdad11d234a31a0a5a49e42fbda99, 87612679f1764898a407f02fbd6a5c94
I20260806 03:52:06.619360 23981 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260806 03:52:07.110648 23991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:07.110338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:41455)
0806 03:52:07.110532 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:07.110587 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":97,"threads_started":1}
I20260806 03:52:07.111011 23992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:07.110737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:37471)
0806 03:52:07.110919 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:07.110958 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":61,"threads_started":1}
I20260806 03:52:07.111451 23993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:07.111230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:38345)
0806 03:52:07.111367 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:07.111408 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":57,"threads_started":1}
I20260806 03:52:08.111425 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.111141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:54253)
0806 03:52:08.111315 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.111365 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260806 03:52:08.111852 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.111653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:37391)
0806 03:52:08.111767 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.111808 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":54,"threads_started":1}
I20260806 03:52:08.112031 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.111796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:41085)
0806 03:52:08.111899 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.111982 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":47,"threads_started":1}
I20260806 03:52:08.267503 23999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:08.267622 23999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } }
I20260806 03:52:08.267794 23999 leader_election.cc:288] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955), 87612679f1764898a407f02fbd6a5c94 (127.12.15.125:42223)
I20260806 03:52:08.268044 22985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2704f24a42474edea3016633c371c5f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "87612679f1764898a407f02fbd6a5c94" is_pre_election: true
I20260806 03:52:08.268174 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.267970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58262)
0806 03:52:08.268078 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.268129 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":57,"threads_started":1}
I20260806 03:52:08.268172 22985 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2704f24a42474edea3016633c371c5f1 in term 2.
I20260806 03:52:08.268353 23223 leader_election.cc:302] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2704f24a42474edea3016633c371c5f1, 87612679f1764898a407f02fbd6a5c94; no voters:
I20260806 03:52:08.268446 23999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260806 03:52:08.268486 23999 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:08.268500 23999 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:52:08.268612 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.268512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58278)
0806 03:52:08.268551 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.268581 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 03:52:08.268788 23222 leader_election.cc:334] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955): Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:08.269374 23999 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } }
I20260806 03:52:08.269500 23999 leader_election.cc:288] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 3 election: Requested vote from peers 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955), 87612679f1764898a407f02fbd6a5c94 (127.12.15.125:42223)
I20260806 03:52:08.269661 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.269575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58282)
0806 03:52:08.269608 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.269635 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 03:52:08.269654 22985 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "2704f24a42474edea3016633c371c5f1" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "87612679f1764898a407f02fbd6a5c94"
I20260806 03:52:08.269724 22985 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:52:08.269935 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.269851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58288)
0806 03:52:08.269889 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.269912 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 03:52:08.270018 23222 leader_election.cc:334] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955): Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
I20260806 03:52:08.270474 22985 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2704f24a42474edea3016633c371c5f1 in term 3.
I20260806 03:52:08.270633 23223 leader_election.cc:302] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2704f24a42474edea3016633c371c5f1, 87612679f1764898a407f02fbd6a5c94; no voters: 2dcfdad11d234a31a0a5a49e42fbda99
I20260806 03:52:08.270725 23999 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:52:08.270820 23999 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [term 3 LEADER]: Becoming Leader. State: Replica: 2704f24a42474edea3016633c371c5f1, State: Running, Role: LEADER
I20260806 03:52:08.270888 23999 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } }
I20260806 03:52:08.271227 24003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2704f24a42474edea3016633c371c5f1. Latest consensus state: current_term: 3 leader_uuid: "2704f24a42474edea3016633c371c5f1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:08.271303 24003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:08.271441 24005 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:08.271957 24005 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:08.272042 24005 catalog_manager.cc:1259] Loaded cluster ID: 2608214565024160a23a9ecdf1abeae3
I20260806 03:52:08.272073 24005 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:52:08.272250 24005 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:52:08.272323 24005 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Loaded TSK: 0
I20260806 03:52:08.272485 24005 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:52:08.290865 23244 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:52:08.292543 23244 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:52:08.293541 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.293441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58306)
0806 03:52:08.293482 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.293511 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 03:52:08.293574 22985 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [term 3 FOLLOWER]: Refusing update from remote peer 2704f24a42474edea3016633c371c5f1: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260806 03:52:08.293828 24003 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260806 03:52:08.294035 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.293783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.0.0.1:58312)
0806 03:52:08.293987 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.294013 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
W20260806 03:52:08.294176 23222 consensus_peers.cc:487] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 -> Peer 2dcfdad11d234a31a0a5a49e42fbda99 (127.12.15.126:40955): Couldn't send request to peer 2dcfdad11d234a31a0a5a49e42fbda99. Status: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260806 03:52:08.294422 24009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2704f24a42474edea3016633c371c5f1. Latest consensus state: current_term: 3 leader_uuid: "2704f24a42474edea3016633c371c5f1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:08.294524 24009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:52:08.294900 24003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "2704f24a42474edea3016633c371c5f1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:08.294984 24003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:08.295135 24010 mvcc.cc:205] Tried to move back new op lower bound from 7315408192688508928 to 7315408192598118400. Current Snapshot: MvccSnapshot[applied={T|T < 7315408192688508928}]
I20260806 03:52:08.295224 24009 sys_catalog.cc:455] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "2704f24a42474edea3016633c371c5f1" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "2dcfdad11d234a31a0a5a49e42fbda99" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40955 } } peers { permanent_uuid: "87612679f1764898a407f02fbd6a5c94" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 42223 } attrs { promote: false } } peers { permanent_uuid: "2704f24a42474edea3016633c371c5f1" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 39491 } attrs { promote: false } } }
I20260806 03:52:08.295305 24009 sys_catalog.cc:458] T 00000000000000000000000000000000 P 87612679f1764898a407f02fbd6a5c94 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:52:08.299558 24000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.296430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:45395 (local address 127.0.0.1:35756)
0806 03:52:08.296459 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.296470 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:08.296535 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:08.296771 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:08.296771 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:08.296773 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:08.296827 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.296829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.297782 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.297783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.298562 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.298564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.299355 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.299356 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.299422 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.299427 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:52:08.299452 (+ 25us) client_negotiation.cc:240] Negotiation successful
0806 03:52:08.299492 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260806 03:52:08.299813 24025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.296640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:43879 (local address 127.0.0.1:36756)
0806 03:52:08.296795 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.296803 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:08.296841 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:08.297158 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:08.297159 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:08.297161 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:08.297216 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.297218 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.297823 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.297823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.299031 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.299033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.299589 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.299590 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.299651 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.299655 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:08.299698 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:52:08.299739 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":43,"thread_start_us":58,"threads_started":1}
I20260806 03:52:08.299891 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.296519 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35756 (local address 127.12.15.67:45395)
0806 03:52:08.296630 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:08.296632 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:08.296639 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:08.296663 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:08.296664 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:08.296674 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:08.296890 (+ 216us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:08.296929 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.297704 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.298652 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.299263 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.299742 (+ 479us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.299759 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:08.299778 (+ 19us) server_negotiation.cc:299] Negotiation successful
0806 03:52:08.299821 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":69,"threads_started":1}
I20260806 03:52:08.300045 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.296595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59062 (local address 127.12.15.66:40675)
0806 03:52:08.296712 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:08.296714 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:08.296835 (+ 121us) server_negotiation.cc:407] Connection header received
0806 03:52:08.296864 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:08.296865 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:08.296875 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:08.296939 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:08.297067 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.297758 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.299141 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.299705 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.299764 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.299775 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:08.299939 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 03:52:08.299981 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":67,"threads_started":1}
I20260806 03:52:08.300050 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.296797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36756 (local address 127.12.15.65:43879)
0806 03:52:08.297016 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:08.297018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:08.297025 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:08.297049 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:08.297050 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:08.297059 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:08.297112 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:08.297273 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.297753 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.299129 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.299518 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.299578 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.299590 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:08.299909 (+ 319us) server_negotiation.cc:299] Negotiation successful
0806 03:52:08.299963 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":63,"threads_started":1}
I20260806 03:52:08.300071 23795 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4d31c7f9114d81b6669c3c10a4e384 (DEFAULT_TABLE table=test-table [id=1aa1ae2c0df64a89b1edee0a1d1e95f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:08.300209 23795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4d31c7f9114d81b6669c3c10a4e384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:08.300251 24020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.296496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:40675 (local address 127.0.0.1:59062)
0806 03:52:08.296716 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.296726 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:08.296781 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:08.296942 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:08.296943 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:08.296945 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:08.296975 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.296976 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.297839 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.297840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.299037 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.299039 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.299806 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.299807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.299861 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.299865 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:08.300145 (+ 280us) client_negotiation.cc:240] Negotiation successful
0806 03:52:08.300189 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"mutex_wait_us":28,"thread_start_us":67,"threads_started":1}
I20260806 03:52:08.300282 23409 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4d31c7f9114d81b6669c3c10a4e384 (DEFAULT_TABLE table=test-table [id=1aa1ae2c0df64a89b1edee0a1d1e95f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:08.300407 23409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4d31c7f9114d81b6669c3c10a4e384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:08.301292 24027 tablet_bootstrap.cc:492] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: Bootstrap starting.
I20260806 03:52:08.301499 24029 tablet_bootstrap.cc:492] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a: Bootstrap starting.
I20260806 03:52:08.301700 24027 tablet_bootstrap.cc:654] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:08.301713 23602 tablet_service.cc:1467] Processing CreateTablet for tablet 5a4d31c7f9114d81b6669c3c10a4e384 (DEFAULT_TABLE table=test-table [id=1aa1ae2c0df64a89b1edee0a1d1e95f1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:08.301827 23602 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a4d31c7f9114d81b6669c3c10a4e384. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:08.301913 24029 tablet_bootstrap.cc:654] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:08.302342 24027 tablet_bootstrap.cc:492] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: No bootstrap required, opened a new log
I20260806 03:52:08.302381 24027 ts_tablet_manager.cc:1397] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:08.302502 24027 raft_consensus.cc:348] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.302546 24027 raft_consensus.cc:374] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:08.302557 24027 raft_consensus.cc:729] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f113aabda2943449519b0a57873f4ce, State: Initialized, Role: FOLLOWER
I20260806 03:52:08.302600 24027 consensus_queue.cc:260] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.302768 24027 ts_tablet_manager.cc:1428] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:08.303020 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.302903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:60357)
0806 03:52:08.302971 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.302995 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 03:52:08.303295 23889 heartbeater.cc:503] Master 127.12.15.124:39491 was elected leader, sending a full tablet report...
I20260806 03:52:08.303347 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.303215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:58685)
0806 03:52:08.303248 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.303280 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 03:52:08.303356 24029 tablet_bootstrap.cc:492] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a: No bootstrap required, opened a new log
I20260806 03:52:08.303404 24029 ts_tablet_manager.cc:1397] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:08.303537 24029 raft_consensus.cc:348] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.303623 24029 raft_consensus.cc:374] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:08.303659 24034 tablet_bootstrap.cc:492] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: Bootstrap starting.
I20260806 03:52:08.303686 24029 raft_consensus.cc:729] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34bcb6df749444de99b1fe01c02d6c6a, State: Initialized, Role: FOLLOWER
I20260806 03:52:08.303736 24029 consensus_queue.cc:260] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.303911 24029 ts_tablet_manager.cc:1428] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:08.304061 24034 tablet_bootstrap.cc:654] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:08.304179 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.304056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:41393)
0806 03:52:08.304126 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.304151 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260806 03:52:08.304528 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.304408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:33151)
0806 03:52:08.304476 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.304500 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260806 03:52:08.304530 23502 heartbeater.cc:503] Master 127.12.15.124:39491 was elected leader, sending a full tablet report...
I20260806 03:52:08.304666 24034 tablet_bootstrap.cc:492] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: No bootstrap required, opened a new log
I20260806 03:52:08.304705 24034 ts_tablet_manager.cc:1397] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:08.304816 24034 raft_consensus.cc:348] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.304876 24034 raft_consensus.cc:374] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:08.304893 24034 raft_consensus.cc:729] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7af4122ee524f78a548e4f2be47996a, State: Initialized, Role: FOLLOWER
I20260806 03:52:08.304934 24034 consensus_queue.cc:260] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.305094 24034 ts_tablet_manager.cc:1428] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:08.305409 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.305280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:37075)
0806 03:52:08.305347 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.305381 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260806 03:52:08.305696 23697 heartbeater.cc:503] Master 127.12.15.124:39491 was elected leader, sending a full tablet report...
I20260806 03:52:08.305757 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.305623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:49545)
0806 03:52:08.305661 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.305692 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 03:52:08.413064 24033 raft_consensus.cc:482] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:08.413170 24033 raft_consensus.cc:504] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.413502 24033 leader_election.cc:288] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675), 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65:43879)
I20260806 03:52:08.416432 24026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.413877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:42483 (local address 127.12.15.65:43879)
0806 03:52:08.413923 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:08.413924 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:08.414017 (+ 93us) server_negotiation.cc:407] Connection header received
0806 03:52:08.414066 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:08.414066 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:08.414075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:08.414153 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:08.414266 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.414720 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.415749 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.416125 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.416196 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.416205 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:08.416323 (+ 118us) server_negotiation.cc:299] Negotiation successful
0806 03:52:08.416364 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:52:08.416441 24024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.413830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:58235 (local address 127.12.15.66:40675)
0806 03:52:08.413880 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:08.413882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:08.413892 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:08.413923 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:08.413924 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:08.413933 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:08.413998 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:08.414223 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.414728 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.415717 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.416098 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.416150 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.416159 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:08.416339 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 03:52:08.416376 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:52:08.416473 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.413754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:43879 (local address 127.12.15.67:42483)
0806 03:52:08.413944 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.413955 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:08.414013 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:08.414154 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:08.414155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:08.414157 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:08.414207 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.414208 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.414790 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.414791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.415662 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.415664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.416204 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.416206 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.416257 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.416260 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:08.416307 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:52:08.416353 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":123,"threads_started":1}
I20260806 03:52:08.416478 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.413640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:40675 (local address 127.12.15.67:58235)
0806 03:52:08.413703 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.413715 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:08.413768 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:08.414032 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:08.414033 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:08.414036 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:08.414145 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.414148 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.414806 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.414807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.415641 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:08.415643 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.416163 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.416164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:08.416249 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.416252 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:08.416296 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:52:08.416358 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 03:52:08.416848 23433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4d31c7f9114d81b6669c3c10a4e384" candidate_uuid: "7f113aabda2943449519b0a57873f4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bcb6df749444de99b1fe01c02d6c6a" is_pre_election: true
I20260806 03:52:08.416880 23631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4d31c7f9114d81b6669c3c10a4e384" candidate_uuid: "7f113aabda2943449519b0a57873f4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7af4122ee524f78a548e4f2be47996a" is_pre_election: true
I20260806 03:52:08.416961 23631 raft_consensus.cc:2454] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f113aabda2943449519b0a57873f4ce in term 0.
I20260806 03:52:08.416955 23433 raft_consensus.cc:2454] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7f113aabda2943449519b0a57873f4ce in term 0.
I20260806 03:52:08.417140 23728 leader_election.cc:302] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 34bcb6df749444de99b1fe01c02d6c6a, 7f113aabda2943449519b0a57873f4ce; no voters:
I20260806 03:52:08.417273 24033 raft_consensus.cc:2781] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:52:08.417322 24033 raft_consensus.cc:482] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:08.417338 24033 raft_consensus.cc:3037] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:08.418015 24033 raft_consensus.cc:504] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.418118 24033 leader_election.cc:288] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [CANDIDATE]: Term 1 election: Requested vote from peers c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675), 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65:43879)
I20260806 03:52:08.418305 23631 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4d31c7f9114d81b6669c3c10a4e384" candidate_uuid: "7f113aabda2943449519b0a57873f4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7af4122ee524f78a548e4f2be47996a"
I20260806 03:52:08.418336 23433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a4d31c7f9114d81b6669c3c10a4e384" candidate_uuid: "7f113aabda2943449519b0a57873f4ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "34bcb6df749444de99b1fe01c02d6c6a"
I20260806 03:52:08.418382 23631 raft_consensus.cc:3037] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:08.418402 23433 raft_consensus.cc:3037] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:08.418977 23631 raft_consensus.cc:2454] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f113aabda2943449519b0a57873f4ce in term 1.
I20260806 03:52:08.419073 23433 raft_consensus.cc:2454] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7f113aabda2943449519b0a57873f4ce in term 1.
I20260806 03:52:08.419137 23728 leader_election.cc:302] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f113aabda2943449519b0a57873f4ce, c7af4122ee524f78a548e4f2be47996a; no voters:
I20260806 03:52:08.419227 24033 raft_consensus.cc:2781] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:08.419369 24033 raft_consensus.cc:686] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 1 LEADER]: Becoming Leader. State: Replica: 7f113aabda2943449519b0a57873f4ce, State: Running, Role: LEADER
I20260806 03:52:08.419427 24033 consensus_queue.cc:237] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } }
I20260806 03:52:08.419706 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.419617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:33769)
0806 03:52:08.419649 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.419677 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260806 03:52:08.420125 23243 catalog_manager.cc:5495] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 7f113aabda2943449519b0a57873f4ce (127.12.15.67). New cstate: current_term: 1 leader_uuid: "7f113aabda2943449519b0a57873f4ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f113aabda2943449519b0a57873f4ce" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 45395 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 } health_report { overall_health: UNKNOWN } } }
I20260806 03:52:08.443003 24023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.440914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35758 (local address 127.12.15.67:45395)
0806 03:52:08.440968 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:08.440970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:08.440979 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:08.441013 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:08.441014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:08.441034 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:08.441094 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:08.441191 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.441690 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.442387 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:08.442594 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:08.442646 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:08.442748 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:08.442848 (+ 100us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:08.442895 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:08.442912 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 03:52:08.442929 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:52:08.444164 23631 raft_consensus.cc:1261] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Refusing update from remote peer 7f113aabda2943449519b0a57873f4ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:08.444207 23433 raft_consensus.cc:1261] T 5a4d31c7f9114d81b6669c3c10a4e384 P 34bcb6df749444de99b1fe01c02d6c6a [term 1 FOLLOWER]: Refusing update from remote peer 7f113aabda2943449519b0a57873f4ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:08.444469 24043 consensus_queue.cc:1035] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7af4122ee524f78a548e4f2be47996a" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:08.444595 24033 consensus_queue.cc:1035] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "34bcb6df749444de99b1fe01c02d6c6a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 43879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:08.445502 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.445369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:55871)
0806 03:52:08.445423 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.445468 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 03:52:08.445581 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.445481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:35495)
0806 03:52:08.445519 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.445549 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 03:52:08.446663 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.446532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:39703)
0806 03:52:08.446599 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.446637 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260806 03:52:08.446929 23997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.446698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.65:44753)
0806 03:52:08.446875 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.446903 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161}
I20260806 03:52:08.447248 23995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.447110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.67:36849)
0806 03:52:08.447199 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.447225 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260806 03:52:08.447885 24049 mvcc.cc:205] Tried to move back new op lower bound from 7315408193305100288 to 7315408193206550528. Current Snapshot: MvccSnapshot[applied={T|T < 7315408193305100288}]
I20260806 03:52:08.448598 23998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:08.448442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40955 (local address 127.12.15.66:49025)
0806 03:52:08.448542 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.448567 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:40955: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260806 03:52:08.448746 23244 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:08.448828 23244 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:08.449940 23244 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2704f24a42474edea3016633c371c5f1: Sending DeleteTablet for 3 replicas of tablet 5a4d31c7f9114d81b6669c3c10a4e384
I20260806 03:52:08.450397 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 23190
I20260806 03:52:08.450488 23795 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4d31c7f9114d81b6669c3c10a4e384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:08 UTC) from {username='oryx', principal='oryx/127.12.15.124@KRBTEST.COM'} at 127.0.0.1:35756
I20260806 03:52:08.450613 23409 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4d31c7f9114d81b6669c3c10a4e384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:08 UTC) from {username='oryx', principal='oryx/127.12.15.124@KRBTEST.COM'} at 127.0.0.1:36756
I20260806 03:52:08.455713 23602 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a4d31c7f9114d81b6669c3c10a4e384 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:08 UTC) from {username='oryx', principal='oryx/127.12.15.124@KRBTEST.COM'} at 127.0.0.1:59062
I20260806 03:52:08.455909 24063 tablet_replica.cc:330] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: stopping tablet replica
I20260806 03:52:08.455966 24063 raft_consensus.cc:2229] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:08.456005 24063 raft_consensus.cc:2258] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:08.456300 24063 ts_tablet_manager.cc:1905] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:08.457737 24063 ts_tablet_manager.cc:1918] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:08.457791 24063 log.cc:1196] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/5a4d31c7f9114d81b6669c3c10a4e384
I20260806 03:52:08.458005 24063 ts_tablet_manager.cc:1939] T 5a4d31c7f9114d81b6669c3c10a4e384 P c7af4122ee524f78a548e4f2be47996a: Deleting consensus metadata
I20260806 03:52:08.458274 24062 tablet_replica.cc:330] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: stopping tablet replica
I20260806 03:52:08.458302 23223 catalog_manager.cc:4841] TS c7af4122ee524f78a548e4f2be47996a (127.12.15.66:40675): tablet 5a4d31c7f9114d81b6669c3c10a4e384 (table test-table [id=1aa1ae2c0df64a89b1edee0a1d1e95f1]) successfully deleted
I20260806 03:52:08.458415 24062 raft_consensus.cc:2229] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:52:08.458540 24062 raft_consensus.cc:2258] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:08.463902 24062 ts_tablet_manager.cc:1905] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 03:52:08.464334 24020 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:08.463879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:43879 (local address 127.0.0.1:36764)
0806 03:52:08.463936 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:08.463948 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:08.464011 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:08.464281 (+ 270us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:43879: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:52:08.464473 23223 catalog_manager.cc:4584] TS 34bcb6df749444de99b1fe01c02d6c6a (127.12.15.65:43879): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a4d31c7f9114d81b6669c3c10a4e384: Network error: Client connection negotiation failed: client connection to 127.12.15.65:43879: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:52:08.464902 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 23514
I20260806 03:52:08.469668 24062 ts_tablet_manager.cc:1918] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:08.469717 24062 log.cc:1196] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/5a4d31c7f9114d81b6669c3c10a4e384
I20260806 03:52:08.469899 24062 ts_tablet_manager.cc:1939] T 5a4d31c7f9114d81b6669c3c10a4e384 P 7f113aabda2943449519b0a57873f4ce: Deleting consensus metadata
I20260806 03:52:08.482409 23223 catalog_manager.cc:4841] TS 7f113aabda2943449519b0a57873f4ce (127.12.15.67:45395): tablet 5a4d31c7f9114d81b6669c3c10a4e384 (table test-table [id=1aa1ae2c0df64a89b1edee0a1d1e95f1]) successfully deleted
I20260806 03:52:08.487097 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 23706
I20260806 03:52:08.495759 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 22779
I20260806 03:52:08.502830 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 23045
2026-08-06T03:52:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6619 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:52:08 dist-test-slave-bqcl krb5kdc[24067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:52:08 dist-test-slave-bqcl krb5kdc[24067](info): set up 2 sockets
Aug 06 03:52:08 dist-test-slave-bqcl krb5kdc[24067](info): commencing operation
krb5kdc: starting...
W20260806 03:52:09.659396 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:52:09 dist-test-slave-bqcl krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:52:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:09Z Disabled control of system clock
WARNING: no policy specified for oryx/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.126.keytab.
Entry for principal oryx/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.126.keytab.
I20260806 03:52:09.774873 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:34289
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.126.keytab
--principal=oryx/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:34289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:09.855683 24090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:09.855897 24090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:09.855930 24090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:09.857101 24090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:09.857147 24090 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260806 03:52:09.857170 24090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:09.857190 24090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:09.857214 24090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:09.857241 24090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:09.858794 24090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:34289
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.126.keytab
--principal=oryx/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:34289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24090
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:09.859050 24090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:09.859282 24090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:09.863884 24097 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:09.863938 24101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:09.864158 24090 server_base.cc:1034] running on GCE node
W20260806 03:52:09.864244 24098 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:09.864503 24090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:09.864791 24090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:09.865932 24090 hybrid_clock.cc:648] HybridClock initialized: now 1785988329865916 us; error 29 us; skew 500 ppm
Aug 06 03:52:09 dist-test-slave-bqcl krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988329, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:09.868361 24090 init.cc:377] Logged in from keytab as oryx/127.12.15.126@KRBTEST.COM (short username oryx)
I20260806 03:52:09.868803 24090 webserver.cc:460] Webserver started at http://127.12.15.126:33405/ using document root <none> and password file <none>
I20260806 03:52:09.869004 24090 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:09.869050 24090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:09.869171 24090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:09.870184 24090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "959beb2c2d5b417daf47b27c92bbbd5f"
format_stamp: "Formatted at 2026-08-06 03:52:09 on dist-test-slave-bqcl"
server_key: "bae6d596061c4b6316f4a1c7fab60a1b"
server_key_iv: "0262fb18343eb886a3ddaa8173ecd1c7"
server_key_version: "encryptionkey@0"
I20260806 03:52:09.870520 24090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "959beb2c2d5b417daf47b27c92bbbd5f"
format_stamp: "Formatted at 2026-08-06 03:52:09 on dist-test-slave-bqcl"
server_key: "bae6d596061c4b6316f4a1c7fab60a1b"
server_key_iv: "0262fb18343eb886a3ddaa8173ecd1c7"
server_key_version: "encryptionkey@0"
I20260806 03:52:09.871718 24090 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:52:09.872340 24109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:09.872498 24090 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:09.872581 24090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "959beb2c2d5b417daf47b27c92bbbd5f"
format_stamp: "Formatted at 2026-08-06 03:52:09 on dist-test-slave-bqcl"
server_key: "bae6d596061c4b6316f4a1c7fab60a1b"
server_key_iv: "0262fb18343eb886a3ddaa8173ecd1c7"
server_key_version: "encryptionkey@0"
I20260806 03:52:09.872659 24090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:09.886900 24090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:09.888569 24090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:09.888711 24090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:09.893138 24090 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:34289
I20260806 03:52:09.893203 24199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:34289 every 8 connection(s)
I20260806 03:52:09.893493 24090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:52:09.894148 24200 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:09.896409 24200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: Bootstrap starting.
I20260806 03:52:09.896987 24200 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:09.897256 24200 log.cc:824] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:09.897912 24200 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: No bootstrap required, opened a new log
I20260806 03:52:09.899121 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 24090
I20260806 03:52:09.899241 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:52:09.899433 12349 external_mini_cluster.cc:1428] Setting key 90ccffbc2c3661493cde8bedd09c2031
I20260806 03:52:09.900291 24200 raft_consensus.cc:348] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959beb2c2d5b417daf47b27c92bbbd5f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34289 } }
I20260806 03:52:09.900430 24200 raft_consensus.cc:374] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:09.900462 24200 raft_consensus.cc:729] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 959beb2c2d5b417daf47b27c92bbbd5f, State: Initialized, Role: FOLLOWER
I20260806 03:52:09.900599 24200 consensus_queue.cc:260] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959beb2c2d5b417daf47b27c92bbbd5f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34289 } }
I20260806 03:52:09.900663 24200 raft_consensus.cc:388] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:52:09.900693 24200 raft_consensus.cc:482] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:52:09.900741 24200 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:09.901233 24200 raft_consensus.cc:504] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959beb2c2d5b417daf47b27c92bbbd5f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34289 } }
I20260806 03:52:09.901328 24200 leader_election.cc:302] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 959beb2c2d5b417daf47b27c92bbbd5f; no voters:
I20260806 03:52:09.901474 24200 leader_election.cc:288] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:52:09.901518 24210 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:09.901624 24210 raft_consensus.cc:686] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [term 1 LEADER]: Becoming Leader. State: Replica: 959beb2c2d5b417daf47b27c92bbbd5f, State: Running, Role: LEADER
I20260806 03:52:09.901734 24200 sys_catalog.cc:564] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:09.901780 24210 consensus_queue.cc:237] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959beb2c2d5b417daf47b27c92bbbd5f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34289 } }
I20260806 03:52:09.902215 24210 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 959beb2c2d5b417daf47b27c92bbbd5f. Latest consensus state: current_term: 1 leader_uuid: "959beb2c2d5b417daf47b27c92bbbd5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959beb2c2d5b417daf47b27c92bbbd5f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34289 } } }
I20260806 03:52:09.902222 24212 sys_catalog.cc:455] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "959beb2c2d5b417daf47b27c92bbbd5f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "959beb2c2d5b417daf47b27c92bbbd5f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34289 } } }
I20260806 03:52:09.902413 24212 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:09.902616 24210 sys_catalog.cc:458] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f [sys.catalog]: This master's current role is: LEADER
W20260806 03:52:09.903456 24233 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:52:09.903512 24233 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:52:09.904264 24220 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:09.904443 24220 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:09.906059 24220 catalog_manager.cc:1347] Generated new cluster ID: 9820b5407b1c4ddaa92f8442d5e14627
I20260806 03:52:09.906104 24220 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:52:09 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988329, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:09.910781 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:09.899971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55718 (local address 127.12.15.126:34289)
0806 03:52:09.900152 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:09.900161 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:09.900186 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:52:09.900284 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:09.900289 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:09.900332 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:09.900446 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:09.901144 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:09.902423 (+ 1279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:09.903468 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:09.904047 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:09.904160 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:09.906809 (+ 2649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:09.906813 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:09.906814 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:09.906825 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:09.909149 (+ 2324us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:09.909553 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:09.909556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:09.909556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:09.909661 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:09.909858 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:09.909859 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:09.909859 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:09.910167 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:09.910311 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:09.910423 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 03:52:09.910554 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":82,"threads_started":1}
I20260806 03:52:09.911983 24220 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:52:09.912451 24220 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:52:09.915246 24220 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: Generated new TSK 0
I20260806 03:52:09.915392 24220 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab.
Entry for principal oryx/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab.
I20260806 03:52:09.940490 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:34289
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab
--principal=oryx/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:10.025813 24239 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:10.026017 24239 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:10.026041 24239 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:10.027208 24239 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:10.027256 24239 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:10.027281 24239 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:52:10.027305 24239 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:10.028805 24239 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab
--principal=oryx/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:34289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24239
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:10.029038 24239 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:10.029258 24239 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:10.034103 24252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:10.034140 24244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:10.034099 24245 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:10.034336 24239 server_base.cc:1034] running on GCE node
I20260806 03:52:10.034545 24239 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:10.034809 24239 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:10.035948 24239 hybrid_clock.cc:648] HybridClock initialized: now 1785988330035935 us; error 29 us; skew 500 ppm
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988330, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:10.038620 24239 init.cc:377] Logged in from keytab as oryx/127.12.15.65@KRBTEST.COM (short username oryx)
I20260806 03:52:10.039158 24239 webserver.cc:460] Webserver started at http://127.12.15.65:46767/ using document root <none> and password file <none>
I20260806 03:52:10.039409 24239 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:10.039518 24239 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:10.039664 24239 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:10.040942 24239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "99453e44a16a4ea2bf6a72b07031e315"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "dff8c97630cd7eb7f4df4848ef9c83ca"
server_key_iv: "ffbd168522e94330960b6b6df9476716"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.041247 24239 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "99453e44a16a4ea2bf6a72b07031e315"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "dff8c97630cd7eb7f4df4848ef9c83ca"
server_key_iv: "ffbd168522e94330960b6b6df9476716"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.042484 24239 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:52:10.043154 24261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.043308 24239 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:10.043394 24239 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "99453e44a16a4ea2bf6a72b07031e315"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "dff8c97630cd7eb7f4df4848ef9c83ca"
server_key_iv: "ffbd168522e94330960b6b6df9476716"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.043466 24239 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:10.054679 24239 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:10.056844 24239 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:10.056983 24239 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:10.057498 24239 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:10.057540 24239 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.057569 24239 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:10.057587 24239 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.063951 24239 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:44223
I20260806 03:52:10.064013 24417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:44223 every 8 connection(s)
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988330, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:10.064414 24239 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:52:10.065093 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 24239
I20260806 03:52:10.065201 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:52:10.065428 12349 external_mini_cluster.cc:1428] Setting key f5d2e35c1ae7549ddef56262c5b6a9e0
I20260806 03:52:10.067695 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.058006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:52607 (local address 127.12.15.126:34289)
0806 03:52:10.058065 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.058067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.058291 (+ 224us) server_negotiation.cc:407] Connection header received
0806 03:52:10.058839 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.058841 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.058856 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.058933 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.059755 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.060549 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.062008 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.062334 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.062414 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.064923 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:10.064924 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:10.064925 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:10.064935 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:10.066490 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.066846 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.066847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.066848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.066913 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.067110 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.067111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.067111 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.067232 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:10.067298 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.067508 (+ 210us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.067594 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:52:10.067830 24274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.057968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:34289 (local address 127.12.15.65:52607)
0806 03:52:10.058156 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.058205 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.058716 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.059006 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.059017 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.059365 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:10.059646 (+ 281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.059652 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.060631 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.060632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.061914 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.061917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.062417 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.062417 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.062525 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.063006 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:10.063024 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:10.064828 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:10.066560 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.066567 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.066568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.066781 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.066971 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.066971 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.066972 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.067063 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.067288 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:10.067291 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:10.067400 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.067475 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.067595 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260806 03:52:10.068313 24418 heartbeater.cc:347] Connected to a master server at 127.12.15.126:34289
I20260806 03:52:10.068413 24418 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:10.068646 24418 heartbeater.cc:511] Master 127.12.15.126:34289 requested a full tablet report, sending...
I20260806 03:52:10.069224 24134 ts_manager.cc:194] Registered new tserver with Master: 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223)
I20260806 03:52:10.069865 24134 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:52607
I20260806 03:52:10.075356 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.069410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:50125 (local address 127.12.15.126:34289)
0806 03:52:10.069448 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.069450 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.070275 (+ 825us) server_negotiation.cc:407] Connection header received
0806 03:52:10.070327 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.070328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.070339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.070394 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.070745 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.071263 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.071984 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.072159 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.072205 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.073074 (+ 869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:10.073074 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:10.073075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:10.073080 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:10.074287 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.074662 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.074663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.074663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.074721 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.074891 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.074891 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.074891 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.074990 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:10.075036 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.075230 (+ 194us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.075295 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:52:10.075378 24431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.069294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:34289 (local address 127.12.15.65:50125)
0806 03:52:10.070213 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.070225 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.070284 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.070397 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.070397 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.070592 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:10.070660 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.070662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.071361 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.071362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.071924 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.071926 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.072225 (+ 299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.072226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.072294 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.072641 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:10.072646 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:10.072998 (+ 352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:10.074357 (+ 1359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.074358 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.074358 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.074600 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.074776 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.074777 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.074777 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.074838 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.075043 (+ 205us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:10.075044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:10.075111 (+ 67us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.075180 (+ 69us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.075279 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":892,"thread_start_us":65,"threads_started":1}
WARNING: no policy specified for oryx/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab.
Entry for principal oryx/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab.
I20260806 03:52:10.092926 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:34289
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab
--principal=oryx/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:10.173734 24434 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:10.173949 24434 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:10.173972 24434 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:10.175143 24434 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:10.175186 24434 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:10.175208 24434 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:52:10.175238 24434 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:10.176788 24434 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab
--principal=oryx/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:34289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24434
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:10.177001 24434 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:10.177222 24434 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:10.182255 24439 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:10.182279 24441 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:10.182350 24434 server_base.cc:1034] running on GCE node
W20260806 03:52:10.182255 24443 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:10.182705 24434 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:10.182912 24434 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:10.184044 24434 hybrid_clock.cc:648] HybridClock initialized: now 1785988330184031 us; error 28 us; skew 500 ppm
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988330, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:10.186544 24434 init.cc:377] Logged in from keytab as oryx/127.12.15.66@KRBTEST.COM (short username oryx)
I20260806 03:52:10.187007 24434 webserver.cc:460] Webserver started at http://127.12.15.66:43541/ using document root <none> and password file <none>
I20260806 03:52:10.187204 24434 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:10.187251 24434 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:10.187377 24434 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:10.188396 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "6fb6f5d5214c4003b5908367856c7ef2"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "30de9d0cc69223a4c511ad6e463ae230"
server_key_iv: "fbc4d7a8d8226e606fa8a096ecea7a52"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.188740 24434 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "6fb6f5d5214c4003b5908367856c7ef2"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "30de9d0cc69223a4c511ad6e463ae230"
server_key_iv: "fbc4d7a8d8226e606fa8a096ecea7a52"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.189946 24434 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:52:10.190660 24453 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.190819 24434 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.190901 24434 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "6fb6f5d5214c4003b5908367856c7ef2"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "30de9d0cc69223a4c511ad6e463ae230"
server_key_iv: "fbc4d7a8d8226e606fa8a096ecea7a52"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.190968 24434 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:10.197942 24434 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:10.200103 24434 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:10.200239 24434 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:10.200757 24434 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:10.200801 24434 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.200830 24434 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:10.200878 24434 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988330, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:10.207837 24434 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:45233
I20260806 03:52:10.208256 24616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:45233 every 8 connection(s)
I20260806 03:52:10.208267 24434 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:52:10.210078 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.201151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:56069 (local address 127.12.15.126:34289)
0806 03:52:10.201204 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.201206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.201411 (+ 205us) server_negotiation.cc:407] Connection header received
0806 03:52:10.202028 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.202029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.202047 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.202100 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.202887 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.203450 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.204542 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.204751 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.204807 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.207654 (+ 2847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:10.207656 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:10.207656 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:10.207665 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:10.209074 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.209405 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.209406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.209406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.209459 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.209618 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.209618 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.209618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.209716 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:10.209759 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.209959 (+ 200us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.210016 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:52:10.210323 24465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.201160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:34289 (local address 127.12.15.66:56069)
0806 03:52:10.201295 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.201331 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.201912 (+ 581us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.202206 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.202210 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.202547 (+ 337us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:10.202768 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.202775 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.203529 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.203530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.204461 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.204464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.204820 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.204821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.204960 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.205478 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:10.205486 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:10.207555 (+ 2069us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:10.209142 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.209146 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.209147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.209353 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.209500 (+ 147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.209500 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.209501 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.209583 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.209757 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:10.209760 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:10.209872 (+ 112us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.209948 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.210065 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":70,"threads_started":1}
I20260806 03:52:10.210784 24617 heartbeater.cc:347] Connected to a master server at 127.12.15.126:34289
I20260806 03:52:10.210863 24617 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:10.211051 24617 heartbeater.cc:511] Master 127.12.15.126:34289 requested a full tablet report, sending...
I20260806 03:52:10.211413 24136 ts_manager.cc:194] Registered new tserver with Master: 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233)
I20260806 03:52:10.211874 24136 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:56069
I20260806 03:52:10.216621 24625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.211795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:34289 (local address 127.12.15.66:41539)
0806 03:52:10.211917 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.211927 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.211985 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.212098 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.212098 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.212278 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:10.212331 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.212332 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.212962 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.212963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.213602 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.213605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.213996 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.213997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.214076 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.214490 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:10.214494 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:10.214765 (+ 271us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:10.215884 (+ 1119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.215885 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.215885 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.216040 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.216172 (+ 132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.216172 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.216172 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.216216 (+ 44us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.216407 (+ 191us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:10.216408 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:10.216441 (+ 33us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.216486 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.216558 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":67,"threads_started":1}
I20260806 03:52:10.216614 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.211854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:41539 (local address 127.12.15.126:34289)
0806 03:52:10.211895 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.211897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.211970 (+ 73us) server_negotiation.cc:407] Connection header received
0806 03:52:10.212029 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.212030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.212040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.212092 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.212397 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.212879 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.213667 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.213915 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.213975 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.214835 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:10.214836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:10.214836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:10.214843 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:10.215815 (+ 972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.216094 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.216094 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.216095 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.216137 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.216247 (+ 110us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.216247 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.216248 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.216366 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:10.216411 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.216501 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.216557 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:52:10.217928 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 24434
I20260806 03:52:10.218020 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:52:10.218245 12349 external_mini_cluster.cc:1428] Setting key 1af4b726ecb8098eef3b87446c10c81a
WARNING: no policy specified for oryx/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab.
Entry for principal oryx/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab.
I20260806 03:52:10.241884 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:34289
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab
--principal=oryx/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40753
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:10.321887 24629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:10.322081 24629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:10.322103 24629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:10.323220 24629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:10.323259 24629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:10.323278 24629 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260806 03:52:10.323297 24629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:10.324714 24629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40753
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab
--principal=oryx/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.12.15.126:34289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:10.324890 24629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:10.325069 24629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:10.329660 24637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:10.329660 24639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:10.329725 24636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:10.329941 24629 server_base.cc:1034] running on GCE node
I20260806 03:52:10.330168 24629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:10.330415 24629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:10.331549 24629 hybrid_clock.cc:648] HybridClock initialized: now 1785988330331536 us; error 25 us; skew 500 ppm
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988330, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:10.333820 24629 init.cc:377] Logged in from keytab as oryx/127.12.15.67@KRBTEST.COM (short username oryx)
I20260806 03:52:10.334244 24629 webserver.cc:460] Webserver started at http://127.12.15.67:43273/ using document root <none> and password file <none>
I20260806 03:52:10.334455 24629 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:10.334502 24629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:10.334623 24629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:10.335749 24629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "c70e07315fc54089ac8d880c6ea968ed"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "da598d3e686dd4653b8199c2a97b5501"
server_key_iv: "c251773bc9e3edf4f0efac7450e1f1a2"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.336120 24629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "c70e07315fc54089ac8d880c6ea968ed"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "da598d3e686dd4653b8199c2a97b5501"
server_key_iv: "c251773bc9e3edf4f0efac7450e1f1a2"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.337329 24629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:10.337981 24647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.338178 24629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:10.338268 24629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "c70e07315fc54089ac8d880c6ea968ed"
format_stamp: "Formatted at 2026-08-06 03:52:10 on dist-test-slave-bqcl"
server_key: "da598d3e686dd4653b8199c2a97b5501"
server_key_iv: "c251773bc9e3edf4f0efac7450e1f1a2"
server_key_version: "encryptionkey@0"
I20260806 03:52:10.338333 24629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:10.353250 24629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:10.355082 24629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:10.355221 24629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:10.355726 24629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:10.355765 24629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.355794 24629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:10.355811 24629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.361889 24629 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:42723
I20260806 03:52:10.361956 24812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:42723 every 8 connection(s)
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988330, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:10.362300 24629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:52:10.364832 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.356159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:42605 (local address 127.12.15.126:34289)
0806 03:52:10.356221 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.356223 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.356420 (+ 197us) server_negotiation.cc:407] Connection header received
0806 03:52:10.356921 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.356922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.356935 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.356986 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.357780 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.358565 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.360024 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.360324 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.360404 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.362789 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:10.362790 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:10.362790 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:10.362797 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:10.363835 (+ 1038us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.364177 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.364178 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.364178 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.364224 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.364382 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.364383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.364383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.364473 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:10.364516 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.364708 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.364769 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:52:10.364996 24663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.356115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:34289 (local address 127.12.15.67:42605)
0806 03:52:10.356298 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.356337 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.356826 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.357053 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.357057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.357399 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:10.357665 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.357673 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.358663 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.358664 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.359927 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.359930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.360405 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.360405 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.360502 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.360961 (+ 459us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:10.360970 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:10.362713 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:10.363899 (+ 1186us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.363905 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.363906 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.364112 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.364260 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.364261 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.364261 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.364343 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.364514 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:10.364516 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:10.364616 (+ 100us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.364695 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.364804 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":74,"threads_started":1}
I20260806 03:52:10.365446 24814 heartbeater.cc:347] Connected to a master server at 127.12.15.126:34289
I20260806 03:52:10.365528 24814 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:10.365721 24814 heartbeater.cc:511] Master 127.12.15.126:34289 requested a full tablet report, sending...
I20260806 03:52:10.366086 24134 ts_manager.cc:194] Registered new tserver with Master: c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723)
I20260806 03:52:10.366381 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 24629
I20260806 03:52:10.366493 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:52:10.366585 24134 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:42605
I20260806 03:52:10.366699 12349 external_mini_cluster.cc:1428] Setting key f073a7144247fe4f11abb3e883517f2b
I20260806 03:52:10.367509 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:52:10.371826 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.366446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:34289 (local address 127.12.15.67:51505)
0806 03:52:10.366595 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.366605 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.366657 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.366775 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.366776 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.366970 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:10.367029 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.367030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.367778 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.367778 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.368469 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.368472 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.368829 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.368830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.368892 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.369212 (+ 320us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:10.369217 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:10.369522 (+ 305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:10.371021 (+ 1499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.371022 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.371022 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.371180 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.371363 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.371363 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.371364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.371407 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.371618 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:10.371619 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:10.371651 (+ 32us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.371696 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.371759 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":69,"threads_started":1}
I20260806 03:52:10.371865 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.366505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:51505 (local address 127.12.15.126:34289)
0806 03:52:10.366544 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.366546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.366660 (+ 114us) server_negotiation.cc:407] Connection header received
0806 03:52:10.366711 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.366712 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.366721 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.366761 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.367087 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.367692 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.368540 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.368763 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.368814 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.369602 (+ 788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:10.369603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:10.369603 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:10.369609 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:10.370946 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.371245 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.371246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.371246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.371304 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.371451 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.371452 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.371452 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.371563 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:10.371628 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.371722 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.371791 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:52:10.389302 24206 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:10.384500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55726 (local address 127.12.15.126:34289)
0806 03:52:10.384558 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.384560 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.384601 (+ 41us) server_negotiation.cc:407] Connection header received
0806 03:52:10.384647 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.384648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.384655 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.384701 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.385210 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.385687 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.386403 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.386577 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.386626 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.389238 (+ 2612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55726: BlockingRecv error: recv got EOF from 127.0.0.1:55726 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988330, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:10.396078 24206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.390068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55730 (local address 127.12.15.126:34289)
0806 03:52:10.390148 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.390150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.390238 (+ 88us) server_negotiation.cc:407] Connection header received
0806 03:52:10.390299 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.390300 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.390310 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.390354 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.390767 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.391224 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.391870 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.392050 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.392101 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.394027 (+ 1926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:10.394028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:10.394028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:10.394034 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:10.395216 (+ 1182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.395495 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.395496 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.395496 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.395539 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.395740 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.395740 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.395740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.395825 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:10.395892 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.395964 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.396016 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:52:10.397459 24134 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:52:10.399085 24134 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:52:10.405413 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.401200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47078 (local address 127.12.15.66:45233)
0806 03:52:10.401396 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.401403 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.401698 (+ 295us) server_negotiation.cc:407] Connection header received
0806 03:52:10.401742 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.401745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.401760 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.401842 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:10.401974 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.403070 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.404507 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.404995 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.405057 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.405132 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.405265 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.405309 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":89,"threads_started":1}
I20260806 03:52:10.405424 24844 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.401109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:45233 (local address 127.0.0.1:47078)
0806 03:52:10.401603 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.401611 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.401656 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.401843 (+ 187us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.401844 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.401846 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:10.401884 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.401885 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.403153 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.403154 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.404392 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.404395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.405072 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.405073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.405152 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.405167 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.405245 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.405313 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"mutex_wait_us":88,"thread_start_us":47,"threads_started":1}
I20260806 03:52:10.405426 24843 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.401096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44223 (local address 127.0.0.1:43352)
0806 03:52:10.401510 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.401518 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.401555 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.401932 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.401933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.401935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:10.401981 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.401982 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.403072 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.403073 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.404159 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.404162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.405089 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.405090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.405168 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.405174 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.405255 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.405322 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":90,"thread_start_us":70,"threads_started":1}
I20260806 03:52:10.405757 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.401095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:42723 (local address 127.0.0.1:44872)
0806 03:52:10.401272 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.401313 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.401378 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.401633 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.401636 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.401638 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:10.401728 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.401730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.403070 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.403071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.404392 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.404395 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.405526 (+ 1131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.405527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.405589 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.405593 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.405649 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.405688 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":35,"thread_start_us":63,"threads_started":1}
I20260806 03:52:10.406031 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.401343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44872 (local address 127.12.15.67:42723)
0806 03:52:10.401477 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.401481 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.401491 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:10.401524 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.401526 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.401539 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.401605 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:10.401795 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.402973 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.404479 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.405170 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.405261 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.405334 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.405907 (+ 573us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.405946 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260806 03:52:10.406319 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.401353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43352 (local address 127.12.15.65:44223)
0806 03:52:10.401761 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.401766 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.401775 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:10.401812 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.401814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.401827 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.402057 (+ 230us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:10.402170 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.402981 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.404248 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.404914 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.404979 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.405053 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.406184 (+ 1131us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.406227 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":85,"threads_started":1}
I20260806 03:52:10.406193 24523 tablet_service.cc:1467] Processing CreateTablet for tablet 44e9091c77964fb3bb30c21a94d88f0f (DEFAULT_TABLE table=test-table [id=ade18e301c764ee0b64e39cd8b45bb27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:10.406592 24523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e9091c77964fb3bb30c21a94d88f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.406750 24720 tablet_service.cc:1467] Processing CreateTablet for tablet 44e9091c77964fb3bb30c21a94d88f0f (DEFAULT_TABLE table=test-table [id=ade18e301c764ee0b64e39cd8b45bb27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:10.406847 24332 tablet_service.cc:1467] Processing CreateTablet for tablet 44e9091c77964fb3bb30c21a94d88f0f (DEFAULT_TABLE table=test-table [id=ade18e301c764ee0b64e39cd8b45bb27]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:10.407089 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e9091c77964fb3bb30c21a94d88f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.407145 24332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 44e9091c77964fb3bb30c21a94d88f0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.409547 24849 tablet_bootstrap.cc:492] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Bootstrap starting.
I20260806 03:52:10.410082 24849 tablet_bootstrap.cc:654] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.410382 24849 log.cc:824] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:10.410913 24850 tablet_bootstrap.cc:492] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Bootstrap starting.
I20260806 03:52:10.411002 24851 tablet_bootstrap.cc:492] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Bootstrap starting.
I20260806 03:52:10.411206 24849 tablet_bootstrap.cc:492] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: No bootstrap required, opened a new log
I20260806 03:52:10.411278 24849 ts_tablet_manager.cc:1397] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:10.411545 24850 tablet_bootstrap.cc:654] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.411561 24851 tablet_bootstrap.cc:654] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.411883 24851 log.cc:824] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:10.411875 24850 log.cc:824] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:10.412645 24850 tablet_bootstrap.cc:492] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: No bootstrap required, opened a new log
I20260806 03:52:10.412664 24851 tablet_bootstrap.cc:492] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: No bootstrap required, opened a new log
I20260806 03:52:10.412714 24850 ts_tablet_manager.cc:1397] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:10.412802 24851 ts_tablet_manager.cc:1397] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:10.413161 24849 raft_consensus.cc:348] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.413332 24849 raft_consensus.cc:374] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.413367 24849 raft_consensus.cc:729] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb6f5d5214c4003b5908367856c7ef2, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.413481 24849 consensus_queue.cc:260] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.413763 24849 ts_tablet_manager.cc:1428] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:52:10.413797 24617 heartbeater.cc:503] Master 127.12.15.126:34289 was elected leader, sending a full tablet report...
I20260806 03:52:10.414532 24850 raft_consensus.cc:348] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.414691 24850 raft_consensus.cc:374] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.414734 24850 raft_consensus.cc:729] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70e07315fc54089ac8d880c6ea968ed, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.414839 24850 consensus_queue.cc:260] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.414815 24851 raft_consensus.cc:348] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.414971 24851 raft_consensus.cc:374] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.415012 24851 raft_consensus.cc:729] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99453e44a16a4ea2bf6a72b07031e315, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.415092 24850 ts_tablet_manager.cc:1428] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 03:52:10.415110 24851 consensus_queue.cc:260] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.415218 24814 heartbeater.cc:503] Master 127.12.15.126:34289 was elected leader, sending a full tablet report...
I20260806 03:52:10.415369 24851 ts_tablet_manager.cc:1428] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:52:10.415395 24418 heartbeater.cc:503] Master 127.12.15.126:34289 was elected leader, sending a full tablet report...
W20260806 03:52:10.459437 24618 tablet.cc:2369] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:52:10.565122 24422 tablet.cc:2369] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:52:10.586473 24862 raft_consensus.cc:482] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:10.586606 24862 raft_consensus.cc:504] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.586978 24862 leader_election.cc:288] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233), 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223)
I20260806 03:52:10.590812 24845 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.587235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:59373 (local address 127.12.15.66:45233)
0806 03:52:10.587279 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.587281 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.587356 (+ 75us) server_negotiation.cc:407] Connection header received
0806 03:52:10.587393 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.587393 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.587401 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.587460 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:10.587668 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.588394 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.589852 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.590458 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.590521 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.590530 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.590705 (+ 175us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.590741 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:52:10.590812 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.587286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:60287 (local address 127.12.15.65:44223)
0806 03:52:10.587333 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.587335 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.587428 (+ 93us) server_negotiation.cc:407] Connection header received
0806 03:52:10.587461 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.587462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.587470 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.587522 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:10.587642 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.588406 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.589852 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.590452 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.590521 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.590529 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.590699 (+ 170us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.590734 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:52:10.590857 24663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.587185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:45233 (local address 127.12.15.67:59373)
0806 03:52:10.587258 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.587274 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.587342 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.587501 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.587502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.587504 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:10.587579 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.587581 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.588467 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.588467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.589772 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.589775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.590535 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.590536 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.590639 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.590641 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.590688 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.590737 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":42}
I20260806 03:52:10.591032 24865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.587185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44223 (local address 127.12.15.67:60287)
0806 03:52:10.587361 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.587370 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.587422 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.587527 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.587528 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.587529 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:10.587584 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.587585 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.588467 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.588467 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.589772 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.589775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.590530 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.590531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.590626 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.590632 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.590689 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.590737 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":72,"thread_start_us":101,"threads_started":1}
I20260806 03:52:10.591231 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e9091c77964fb3bb30c21a94d88f0f" candidate_uuid: "c70e07315fc54089ac8d880c6ea968ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb6f5d5214c4003b5908367856c7ef2" is_pre_election: true
I20260806 03:52:10.591401 24556 raft_consensus.cc:2454] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70e07315fc54089ac8d880c6ea968ed in term 0.
I20260806 03:52:10.591421 24359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e9091c77964fb3bb30c21a94d88f0f" candidate_uuid: "c70e07315fc54089ac8d880c6ea968ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99453e44a16a4ea2bf6a72b07031e315" is_pre_election: true
I20260806 03:52:10.591555 24359 raft_consensus.cc:2454] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70e07315fc54089ac8d880c6ea968ed in term 0.
I20260806 03:52:10.591679 24653 leader_election.cc:302] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb6f5d5214c4003b5908367856c7ef2, c70e07315fc54089ac8d880c6ea968ed; no voters:
I20260806 03:52:10.591814 24862 raft_consensus.cc:2781] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:52:10.591873 24862 raft_consensus.cc:482] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:10.591897 24862 raft_consensus.cc:3037] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:10.592823 24862 raft_consensus.cc:504] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.592940 24862 leader_election.cc:288] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [CANDIDATE]: Term 1 election: Requested vote from peers 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233), 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223)
I20260806 03:52:10.593091 24359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e9091c77964fb3bb30c21a94d88f0f" candidate_uuid: "c70e07315fc54089ac8d880c6ea968ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99453e44a16a4ea2bf6a72b07031e315"
I20260806 03:52:10.593093 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "44e9091c77964fb3bb30c21a94d88f0f" candidate_uuid: "c70e07315fc54089ac8d880c6ea968ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb6f5d5214c4003b5908367856c7ef2"
I20260806 03:52:10.593168 24556 raft_consensus.cc:3037] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:10.593171 24359 raft_consensus.cc:3037] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:10.593822 24359 raft_consensus.cc:2454] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70e07315fc54089ac8d880c6ea968ed in term 1.
I20260806 03:52:10.593897 24556 raft_consensus.cc:2454] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70e07315fc54089ac8d880c6ea968ed in term 1.
I20260806 03:52:10.594026 24650 leader_election.cc:302] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99453e44a16a4ea2bf6a72b07031e315, c70e07315fc54089ac8d880c6ea968ed; no voters:
I20260806 03:52:10.594141 24862 raft_consensus.cc:2781] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:10.594282 24862 raft_consensus.cc:686] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 1 LEADER]: Becoming Leader. State: Replica: c70e07315fc54089ac8d880c6ea968ed, State: Running, Role: LEADER
I20260806 03:52:10.594379 24862 consensus_queue.cc:237] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } }
I20260806 03:52:10.595237 24134 catalog_manager.cc:5495] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed reported cstate change: term changed from 0 to 1, leader changed from <none> to c70e07315fc54089ac8d880c6ea968ed (127.12.15.67). New cstate: current_term: 1 leader_uuid: "c70e07315fc54089ac8d880c6ea968ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } health_report { overall_health: UNKNOWN } } }
W20260806 03:52:10.612875 24815 tablet.cc:2369] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:52:10.628870 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.626661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.12.15.67:42723)
0806 03:52:10.626720 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.626722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.626731 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:10.626766 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.626767 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.626784 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.626847 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:10.626971 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.627436 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.628175 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.628367 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.628422 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.628578 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:10.628679 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:10.628750 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.628775 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.628796 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:52:10.630549 24556 raft_consensus.cc:1261] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Refusing update from remote peer c70e07315fc54089ac8d880c6ea968ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:10.630548 24359 raft_consensus.cc:1261] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Refusing update from remote peer c70e07315fc54089ac8d880c6ea968ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:10.630878 24862 consensus_queue.cc:1035] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:10.631021 24866 consensus_queue.cc:1035] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:10.633937 24873 mvcc.cc:205] Tried to move back new op lower bound from 7315408202260086784 to 7315408202115239936. Current Snapshot: MvccSnapshot[applied={T|T < 7315408202260086784}]
I20260806 03:52:10.633936 24872 mvcc.cc:205] Tried to move back new op lower bound from 7315408202260086784 to 7315408202115239936. Current Snapshot: MvccSnapshot[applied={T|T < 7315408202260086784 or (T in {7315408202260086784})}]
I20260806 03:52:10.635337 24136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:10.635416 24136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:10.636099 24136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: Sending DeleteTablet for 3 replicas of tablet 44e9091c77964fb3bb30c21a94d88f0f
I20260806 03:52:10.636400 24720 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e9091c77964fb3bb30c21a94d88f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:10 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:44872
I20260806 03:52:10.636520 24523 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e9091c77964fb3bb30c21a94d88f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:10 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:47078
I20260806 03:52:10.636632 24885 tablet_replica.cc:330] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: stopping tablet replica
I20260806 03:52:10.636739 24885 raft_consensus.cc:2229] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:52:10.636747 24332 tablet_service.cc:1514] Processing DeleteTablet for tablet 44e9091c77964fb3bb30c21a94d88f0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:10 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:43352
I20260806 03:52:10.636873 24885 raft_consensus.cc:2258] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:10.636943 24887 tablet_replica.cc:330] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: stopping tablet replica
I20260806 03:52:10.636933 24136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55730:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 03:52:10.637041 24887 raft_consensus.cc:2229] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
W20260806 03:52:10.637071 24136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:52:10.637148 24887 raft_consensus.cc:2258] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:10.637584 24887 ts_tablet_manager.cc:1905] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:10.637679 24885 ts_tablet_manager.cc:1905] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:10.638368 24332 tablet_service.cc:1467] Processing CreateTablet for tablet 454fd0195330472c88b240a7f2192830 (DEFAULT_TABLE table=test-table [id=92bd3b30b58348fdbdb201731804a366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:10.638489 24332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 454fd0195330472c88b240a7f2192830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.638576 24720 tablet_service.cc:1467] Processing CreateTablet for tablet 454fd0195330472c88b240a7f2192830 (DEFAULT_TABLE table=test-table [id=92bd3b30b58348fdbdb201731804a366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:10.638680 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 454fd0195330472c88b240a7f2192830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.639571 24885 ts_tablet_manager.cc:1918] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:10.639631 24885 log.cc:1196] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/44e9091c77964fb3bb30c21a94d88f0f
I20260806 03:52:10.639808 24523 tablet_service.cc:1467] Processing CreateTablet for tablet 454fd0195330472c88b240a7f2192830 (DEFAULT_TABLE table=test-table [id=92bd3b30b58348fdbdb201731804a366]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:10.639853 24850 tablet_bootstrap.cc:492] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: Bootstrap starting.
I20260806 03:52:10.639889 24885 ts_tablet_manager.cc:1939] T 44e9091c77964fb3bb30c21a94d88f0f P c70e07315fc54089ac8d880c6ea968ed: Deleting consensus metadata
I20260806 03:52:10.639926 24523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 454fd0195330472c88b240a7f2192830. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.640266 24850 tablet_bootstrap.cc:654] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.640379 24113 catalog_manager.cc:4841] TS c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723): tablet 44e9091c77964fb3bb30c21a94d88f0f (table test-table [id=ade18e301c764ee0b64e39cd8b45bb27]) successfully deleted
I20260806 03:52:10.640511 24888 tablet_replica.cc:330] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: stopping tablet replica
I20260806 03:52:10.640640 24888 raft_consensus.cc:2229] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:10.640751 24888 raft_consensus.cc:2258] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:10.641098 24850 tablet_bootstrap.cc:492] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: No bootstrap required, opened a new log
I20260806 03:52:10.641144 24850 ts_tablet_manager.cc:1397] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:52:10.641142 24888 ts_tablet_manager.cc:1905] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:10.641191 24851 tablet_bootstrap.cc:492] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: Bootstrap starting.
I20260806 03:52:10.641304 24850 raft_consensus.cc:348] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.641382 24850 raft_consensus.cc:374] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.641402 24850 raft_consensus.cc:729] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70e07315fc54089ac8d880c6ea968ed, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.641431 24850 consensus_queue.cc:260] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.641510 24850 ts_tablet_manager.cc:1428] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.641548 24851 tablet_bootstrap.cc:654] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.641999 24849 tablet_bootstrap.cc:492] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: Bootstrap starting.
I20260806 03:52:10.642643 24888 ts_tablet_manager.cc:1918] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:10.642716 24888 log.cc:1196] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/44e9091c77964fb3bb30c21a94d88f0f
I20260806 03:52:10.642756 24849 tablet_bootstrap.cc:654] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.642938 24888 ts_tablet_manager.cc:1939] T 44e9091c77964fb3bb30c21a94d88f0f P 99453e44a16a4ea2bf6a72b07031e315: Deleting consensus metadata
I20260806 03:52:10.643371 24849 tablet_bootstrap.cc:492] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: No bootstrap required, opened a new log
I20260806 03:52:10.643433 24849 ts_tablet_manager.cc:1397] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:10.643599 24849 raft_consensus.cc:348] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.643675 24849 raft_consensus.cc:374] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.643704 24849 raft_consensus.cc:729] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb6f5d5214c4003b5908367856c7ef2, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.643757 24849 consensus_queue.cc:260] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.643849 24849 ts_tablet_manager.cc:1428] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:10.644023 24110 catalog_manager.cc:4841] TS 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223): tablet 44e9091c77964fb3bb30c21a94d88f0f (table test-table [id=ade18e301c764ee0b64e39cd8b45bb27]) successfully deleted
I20260806 03:52:10.644987 24887 ts_tablet_manager.cc:1918] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:10.645069 24887 log.cc:1196] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/44e9091c77964fb3bb30c21a94d88f0f
I20260806 03:52:10.645359 24887 ts_tablet_manager.cc:1939] T 44e9091c77964fb3bb30c21a94d88f0f P 6fb6f5d5214c4003b5908367856c7ef2: Deleting consensus metadata
I20260806 03:52:10.645771 24116 catalog_manager.cc:4841] TS 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233): tablet 44e9091c77964fb3bb30c21a94d88f0f (table test-table [id=ade18e301c764ee0b64e39cd8b45bb27]) successfully deleted
I20260806 03:52:10.646047 24851 tablet_bootstrap.cc:492] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: No bootstrap required, opened a new log
I20260806 03:52:10.646090 24851 ts_tablet_manager.cc:1397] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 03:52:10.646243 24851 raft_consensus.cc:348] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.646297 24851 raft_consensus.cc:374] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.646315 24851 raft_consensus.cc:729] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99453e44a16a4ea2bf6a72b07031e315, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.646354 24851 consensus_queue.cc:260] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.646682 24851 ts_tablet_manager.cc:1428] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260806 03:52:10.875385 24858 raft_consensus.cc:482] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:10.875492 24858 raft_consensus.cc:504] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.875836 24858 leader_election.cc:288] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223), c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723)
I20260806 03:52:10.878671 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.876102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:34895 (local address 127.12.15.65:44223)
0806 03:52:10.876161 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.876164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.876261 (+ 97us) server_negotiation.cc:407] Connection header received
0806 03:52:10.876317 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.876318 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.876327 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.876375 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:10.876484 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.876976 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.877968 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.878363 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.878420 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.878430 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.878564 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.878599 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:52:10.878695 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.876131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:35345 (local address 127.12.15.67:42723)
0806 03:52:10.876178 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.876180 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.876192 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:52:10.876282 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.876283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.876296 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.876349 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:10.876531 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.876997 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.877961 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.878374 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.878434 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.878443 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.878589 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.878633 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:52:10.878707 24894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.876007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44223 (local address 127.12.15.66:34895)
0806 03:52:10.876177 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.876188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.876266 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.876379 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.876380 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.876382 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:10.876439 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.876441 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.877047 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.877048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.877878 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.877881 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.878430 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.878431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.878501 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.878504 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.878543 (+ 39us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.878582 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":40,"thread_start_us":45,"threads_started":1}
I20260806 03:52:10.879025 24359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "454fd0195330472c88b240a7f2192830" candidate_uuid: "6fb6f5d5214c4003b5908367856c7ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99453e44a16a4ea2bf6a72b07031e315" is_pre_election: true
I20260806 03:52:10.878712 24895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.876007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:42723 (local address 127.12.15.66:35345)
0806 03:52:10.876133 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.876145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.876216 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.876359 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.876360 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.876363 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:10.876467 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.876470 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.877073 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.877074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.877878 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.877880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.878456 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.878457 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.878525 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.878528 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.878577 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.878615 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"mutex_wait_us":40,"thread_start_us":61,"threads_started":1}
I20260806 03:52:10.879110 24359 raft_consensus.cc:2454] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fb6f5d5214c4003b5908367856c7ef2 in term 0.
I20260806 03:52:10.879325 24454 leader_election.cc:302] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb6f5d5214c4003b5908367856c7ef2, 99453e44a16a4ea2bf6a72b07031e315; no voters:
I20260806 03:52:10.879312 24750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "454fd0195330472c88b240a7f2192830" candidate_uuid: "6fb6f5d5214c4003b5908367856c7ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70e07315fc54089ac8d880c6ea968ed" is_pre_election: true
I20260806 03:52:10.879416 24750 raft_consensus.cc:2454] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6fb6f5d5214c4003b5908367856c7ef2 in term 0.
I20260806 03:52:10.879453 24858 raft_consensus.cc:2781] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:52:10.879487 24858 raft_consensus.cc:482] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:10.879505 24858 raft_consensus.cc:3037] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:10.880177 24858 raft_consensus.cc:504] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.880302 24858 leader_election.cc:288] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [CANDIDATE]: Term 1 election: Requested vote from peers 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223), c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723)
I20260806 03:52:10.880447 24359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "454fd0195330472c88b240a7f2192830" candidate_uuid: "6fb6f5d5214c4003b5908367856c7ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99453e44a16a4ea2bf6a72b07031e315"
I20260806 03:52:10.880460 24750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "454fd0195330472c88b240a7f2192830" candidate_uuid: "6fb6f5d5214c4003b5908367856c7ef2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70e07315fc54089ac8d880c6ea968ed"
I20260806 03:52:10.880517 24359 raft_consensus.cc:3037] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:10.880532 24750 raft_consensus.cc:3037] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:10.881142 24359 raft_consensus.cc:2454] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fb6f5d5214c4003b5908367856c7ef2 in term 1.
I20260806 03:52:10.881270 24750 raft_consensus.cc:2454] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6fb6f5d5214c4003b5908367856c7ef2 in term 1.
I20260806 03:52:10.881292 24454 leader_election.cc:302] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb6f5d5214c4003b5908367856c7ef2, 99453e44a16a4ea2bf6a72b07031e315; no voters:
I20260806 03:52:10.881362 24858 raft_consensus.cc:2781] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:10.881458 24858 raft_consensus.cc:686] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 LEADER]: Becoming Leader. State: Replica: 6fb6f5d5214c4003b5908367856c7ef2, State: Running, Role: LEADER
I20260806 03:52:10.881527 24858 consensus_queue.cc:237] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.882228 24136 catalog_manager.cc:5495] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66). New cstate: current_term: 1 leader_uuid: "6fb6f5d5214c4003b5908367856c7ef2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } health_report { overall_health: UNKNOWN } } }
Aug 06 03:52:10 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988329, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.126@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:10.953037 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.946852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:34289 (local address 127.0.0.1:55740)
0806 03:52:10.947042 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:10.947054 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:10.947122 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:10.947256 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:10.947257 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:10.947577 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:10.947624 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.947626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.948237 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.948237 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.948764 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:10.948765 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.949102 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.949102 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:10.949137 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.949587 (+ 450us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:10.949591 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:10.950957 (+ 1366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:10.952207 (+ 1250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.952214 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.952216 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.952413 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.952558 (+ 145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:10.952558 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:10.952559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:10.952608 (+ 49us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:10.952784 (+ 176us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:10.952789 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:10.952826 (+ 37us) client_negotiation.cc:769] Sending connection context
0806 03:52:10.952872 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:52:10.952955 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":90,"threads_started":1}
I20260806 03:52:10.953104 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:10.946925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55740 (local address 127.12.15.126:34289)
0806 03:52:10.947074 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:10.947075 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:10.947106 (+ 31us) server_negotiation.cc:407] Connection header received
0806 03:52:10.947174 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:10.947175 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:10.947184 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:10.947245 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:10.947687 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.948182 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.948842 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:10.949042 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:10.949094 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:10.951032 (+ 1938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:10.951033 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:10.951033 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:10.951039 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:10.952138 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.952467 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.952467 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.952468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.952516 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:10.952647 (+ 131us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:10.952648 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:10.952648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:10.952745 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:10.952788 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:10.952895 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 03:52:10.953001 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260806 03:52:10.954895 24136 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:55740:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260806 03:52:10.955116 24136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:52:10.956513 24332 tablet_service.cc:1467] Processing CreateTablet for tablet 89bc31bd85604c2d8e1bb863319c05cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7bc47ce003074ed7a0df2a445bf47704]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:52:10.956506 24523 tablet_service.cc:1467] Processing CreateTablet for tablet 89bc31bd85604c2d8e1bb863319c05cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7bc47ce003074ed7a0df2a445bf47704]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:52:10.956532 24720 tablet_service.cc:1467] Processing CreateTablet for tablet 89bc31bd85604c2d8e1bb863319c05cf (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7bc47ce003074ed7a0df2a445bf47704]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260806 03:52:10.956664 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bc31bd85604c2d8e1bb863319c05cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.956664 24523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bc31bd85604c2d8e1bb863319c05cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.956669 24332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89bc31bd85604c2d8e1bb863319c05cf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:10.957801 24851 tablet_bootstrap.cc:492] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: Bootstrap starting.
I20260806 03:52:10.958076 24849 tablet_bootstrap.cc:492] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: Bootstrap starting.
I20260806 03:52:10.958195 24851 tablet_bootstrap.cc:654] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.958503 24849 tablet_bootstrap.cc:654] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.958832 24850 tablet_bootstrap.cc:492] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: Bootstrap starting.
I20260806 03:52:10.958866 24851 tablet_bootstrap.cc:492] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: No bootstrap required, opened a new log
I20260806 03:52:10.958905 24851 ts_tablet_manager.cc:1397] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:52:10.959055 24851 raft_consensus.cc:348] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.959131 24851 raft_consensus.cc:374] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.959159 24851 raft_consensus.cc:729] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99453e44a16a4ea2bf6a72b07031e315, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.959179 24850 tablet_bootstrap.cc:654] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:10.959209 24851 consensus_queue.cc:260] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.959327 24851 ts_tablet_manager.cc:1428] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.959383 24864 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:10.959479 24864 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:52:10.959525 24849 tablet_bootstrap.cc:492] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: No bootstrap required, opened a new log
I20260806 03:52:10.959575 24849 ts_tablet_manager.cc:1397] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:10.959710 24849 raft_consensus.cc:348] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.959779 24849 raft_consensus.cc:374] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.959798 24849 raft_consensus.cc:729] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb6f5d5214c4003b5908367856c7ef2, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.959842 24849 consensus_queue.cc:260] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.959923 24849 ts_tablet_manager.cc:1428] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.960009 24858 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:10.960099 24858 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:52:10.960332 24850 tablet_bootstrap.cc:492] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: No bootstrap required, opened a new log
I20260806 03:52:10.960381 24850 ts_tablet_manager.cc:1397] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:52:10.960474 24850 raft_consensus.cc:348] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.960543 24850 raft_consensus.cc:374] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:10.960570 24850 raft_consensus.cc:729] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70e07315fc54089ac8d880c6ea968ed, State: Initialized, Role: FOLLOWER
I20260806 03:52:10.960611 24850 consensus_queue.cc:260] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:10.960712 24850 ts_tablet_manager.cc:1428] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:10.960758 24876 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:10.960843 24876 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:52:11.097987 24864 raft_consensus.cc:482] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:11.098115 24864 raft_consensus.cc:504] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.098497 24864 leader_election.cc:288] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233), c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723)
I20260806 03:52:11.102600 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.098745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:44327 (local address 127.12.15.66:45233)
0806 03:52:11.098872 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.098874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.098981 (+ 107us) server_negotiation.cc:407] Connection header received
0806 03:52:11.099034 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.099036 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.099049 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.099110 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:11.099320 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.100078 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.101579 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.102211 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.102269 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.102278 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.102476 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.102517 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":62,"threads_started":1}
I20260806 03:52:11.102599 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.098745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:55779 (local address 127.12.15.67:42723)
0806 03:52:11.098796 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.098799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.098964 (+ 165us) server_negotiation.cc:407] Connection header received
0806 03:52:11.099040 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.099041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.099050 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.099109 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:11.099315 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.100079 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.101579 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.102198 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.102268 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.102278 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.102476 (+ 198us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.102517 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:52:11.102651 24916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.098684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:42723 (local address 127.12.15.65:55779)
0806 03:52:11.098877 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:11.098887 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:11.098970 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:11.099141 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:11.099142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:11.099145 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:11.099252 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.099254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.100151 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.100151 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.101495 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.101499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.102297 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.102298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.102410 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.102417 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 03:52:11.102470 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:52:11.102530 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":51,"thread_start_us":109,"threads_started":1}
I20260806 03:52:11.102651 24915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.098684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:45233 (local address 127.12.15.65:44327)
0806 03:52:11.098875 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:11.098887 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:11.098970 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:11.099141 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:11.099142 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:11.099146 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:11.099240 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.099243 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.100178 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.100179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.101495 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.101499 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.102295 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.102296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.102410 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.102417 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 03:52:11.102469 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 03:52:11.102530 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"mutex_wait_us":65,"thread_start_us":104,"threads_started":1}
I20260806 03:52:11.102936 24750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bc31bd85604c2d8e1bb863319c05cf" candidate_uuid: "99453e44a16a4ea2bf6a72b07031e315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70e07315fc54089ac8d880c6ea968ed" is_pre_election: true
I20260806 03:52:11.103032 24750 raft_consensus.cc:2454] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99453e44a16a4ea2bf6a72b07031e315 in term 0.
I20260806 03:52:11.103228 24263 leader_election.cc:302] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99453e44a16a4ea2bf6a72b07031e315, c70e07315fc54089ac8d880c6ea968ed; no voters:
I20260806 03:52:11.103372 24864 raft_consensus.cc:2781] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:52:11.103448 24864 raft_consensus.cc:482] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:11.103475 24864 raft_consensus.cc:3037] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.103482 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bc31bd85604c2d8e1bb863319c05cf" candidate_uuid: "99453e44a16a4ea2bf6a72b07031e315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb6f5d5214c4003b5908367856c7ef2" is_pre_election: true
I20260806 03:52:11.103569 24556 raft_consensus.cc:2454] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99453e44a16a4ea2bf6a72b07031e315 in term 0.
I20260806 03:52:11.104209 24864 raft_consensus.cc:504] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.104329 24864 leader_election.cc:288] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [CANDIDATE]: Term 1 election: Requested vote from peers 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233), c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723)
I20260806 03:52:11.104473 24750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bc31bd85604c2d8e1bb863319c05cf" candidate_uuid: "99453e44a16a4ea2bf6a72b07031e315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70e07315fc54089ac8d880c6ea968ed"
I20260806 03:52:11.104494 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89bc31bd85604c2d8e1bb863319c05cf" candidate_uuid: "99453e44a16a4ea2bf6a72b07031e315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb6f5d5214c4003b5908367856c7ef2"
I20260806 03:52:11.104545 24750 raft_consensus.cc:3037] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.104553 24556 raft_consensus.cc:3037] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.105105 24556 raft_consensus.cc:2454] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99453e44a16a4ea2bf6a72b07031e315 in term 1.
I20260806 03:52:11.105145 24750 raft_consensus.cc:2454] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99453e44a16a4ea2bf6a72b07031e315 in term 1.
I20260806 03:52:11.105302 24265 leader_election.cc:302] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb6f5d5214c4003b5908367856c7ef2, 99453e44a16a4ea2bf6a72b07031e315; no voters:
I20260806 03:52:11.105396 24864 raft_consensus.cc:2781] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:11.105559 24864 raft_consensus.cc:686] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [term 1 LEADER]: Becoming Leader. State: Replica: 99453e44a16a4ea2bf6a72b07031e315, State: Running, Role: LEADER
I20260806 03:52:11.105638 24864 consensus_queue.cc:237] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.106166 24918 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: TxnStatusTablet state changed. Reason: New leader 99453e44a16a4ea2bf6a72b07031e315. Latest consensus state: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:11.106254 24918 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:52:11.106338 24134 catalog_manager.cc:5495] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } health_report { overall_health: UNKNOWN } } }
I20260806 03:52:11.106418 24920 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260806 03:52:11.106514 24920 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260806 03:52:11.141919 24907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.138991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44223 (local address 127.0.0.1:43368)
0806 03:52:11.139076 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:11.139089 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:11.139133 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:11.139297 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:11.139298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:11.139299 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:52:11.139357 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.139360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.140256 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.140257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.140895 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.140898 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.141380 (+ 482us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.141381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.141434 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.141452 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.141750 (+ 298us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.141762 (+ 12us) client_negotiation.cc:769] Sending connection context
0806 03:52:11.141804 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 03:52:11.141848 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 03:52:11.141939 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.139046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43368 (local address 127.12.15.65:44223)
0806 03:52:11.139101 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.139103 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.139124 (+ 21us) server_negotiation.cc:407] Connection header received
0806 03:52:11.139202 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.139203 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.139225 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.139296 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:11.139410 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.140174 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.140966 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.141287 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.141359 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.141542 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.141676 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.141747 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.141809 (+ 62us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.141831 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:52:11.143224 24556 raft_consensus.cc:1261] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Refusing update from remote peer 99453e44a16a4ea2bf6a72b07031e315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:11.143280 24750 raft_consensus.cc:1261] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Refusing update from remote peer 99453e44a16a4ea2bf6a72b07031e315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:11.143501 24864 consensus_queue.cc:1035] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:11.143597 24918 consensus_queue.cc:1035] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:11.144176 24858 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: TxnStatusTablet state changed. Reason: New leader 99453e44a16a4ea2bf6a72b07031e315. Latest consensus state: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:11.144289 24858 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:52:11.144630 24876 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: TxnStatusTablet state changed. Reason: New leader 99453e44a16a4ea2bf6a72b07031e315. Latest consensus state: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:11.144716 24876 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:52:11.145259 24932 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:11.145370 24932 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:52:11.145455 24864 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:11.145506 24864 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P 99453e44a16a4ea2bf6a72b07031e315: This TxnStatusTablet replica's current role is: LEADER
I20260806 03:52:11.145929 24858 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:11.146001 24858 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P 6fb6f5d5214c4003b5908367856c7ef2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:52:11.146513 24876 tablet_replica.cc:439] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } } }
I20260806 03:52:11.146601 24876 tablet_replica.cc:442] T 89bc31bd85604c2d8e1bb863319c05cf P c70e07315fc54089ac8d880c6ea968ed: This TxnStatusTablet replica's current role is: FOLLOWER
I20260806 03:52:11.147199 24929 mvcc.cc:205] Tried to move back new op lower bound from 7315408204360491008 to 7315408204209344512. Current Snapshot: MvccSnapshot[applied={T|T < 7315408204360491008 or (T in {7315408204360491008})}]
I20260806 03:52:11.150695 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.148152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47088 (local address 127.12.15.66:45233)
0806 03:52:11.148193 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.148195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.148232 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:52:11.148300 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.148301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.148322 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.148401 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:11.148480 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.149204 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.149893 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.150233 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.150293 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.150398 (+ 105us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.150522 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.150589 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.150604 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.150624 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:52:11.155406 24945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.152847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44223 (local address 127.12.15.66:41777)
0806 03:52:11.152984 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:11.152994 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:11.153047 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:11.153187 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:11.153187 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:11.153189 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:52:11.153224 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.153225 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.154031 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.154031 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.154645 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.154647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.155089 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.155090 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.155128 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.155142 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.155281 (+ 139us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.155282 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:52:11.155311 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 03:52:11.155346 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":76,"threads_started":1}
I20260806 03:52:11.155408 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.152913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:41777 (local address 127.12.15.65:44223)
0806 03:52:11.152957 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.152959 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.153048 (+ 89us) server_negotiation.cc:407] Connection header received
0806 03:52:11.153103 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.153104 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.153114 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.153186 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:11.153258 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.153968 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.154711 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.155011 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.155074 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.155201 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.155234 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.155282 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.155314 (+ 32us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.155334 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:52:11.157743 24750 raft_consensus.cc:1261] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Refusing update from remote peer 6fb6f5d5214c4003b5908367856c7ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:11.157779 24359 raft_consensus.cc:1261] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Refusing update from remote peer 6fb6f5d5214c4003b5908367856c7ef2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:11.157981 24858 consensus_queue.cc:1035] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:11.158139 24896 consensus_queue.cc:1035] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:11.166810 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.163879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:45233 (local address 127.12.15.65:41877)
0806 03:52:11.164042 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:11.164053 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:11.164106 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:11.164217 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:11.164217 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:11.164219 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:52:11.164257 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.164258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.165112 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.165113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.165981 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.165984 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.166435 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.166436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.166478 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.166496 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.166651 (+ 155us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.166653 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:52:11.166703 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 03:52:11.166738 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260806 03:52:11.166802 24917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.163924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:41877 (local address 127.12.15.66:45233)
0806 03:52:11.163966 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.163967 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.164107 (+ 140us) server_negotiation.cc:407] Connection header received
0806 03:52:11.164146 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.164147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.164156 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.164215 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:11.164291 (+ 76us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.165047 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.166047 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.166355 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.166420 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.166565 (+ 145us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.166599 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.166646 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.166707 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.166730 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:52:11.175681 24136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:11.175791 24136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55730:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:11.176347 24136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: Sending DeleteTablet for 3 replicas of tablet 454fd0195330472c88b240a7f2192830
I20260806 03:52:11.176581 24332 tablet_service.cc:1514] Processing DeleteTablet for tablet 454fd0195330472c88b240a7f2192830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:43352
I20260806 03:52:11.176635 24523 tablet_service.cc:1514] Processing DeleteTablet for tablet 454fd0195330472c88b240a7f2192830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:47078
I20260806 03:52:11.176931 24720 tablet_service.cc:1514] Processing DeleteTablet for tablet 454fd0195330472c88b240a7f2192830 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:44872
I20260806 03:52:11.177162 24958 tablet_replica.cc:330] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: stopping tablet replica
I20260806 03:52:11.177264 24958 raft_consensus.cc:2229] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:11.177347 24958 raft_consensus.cc:2258] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.177639 24958 ts_tablet_manager.cc:1905] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:11.179526 24958 ts_tablet_manager.cc:1918] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:52:11.179657 24958 log.cc:1196] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/454fd0195330472c88b240a7f2192830
I20260806 03:52:11.179970 24958 ts_tablet_manager.cc:1939] T 454fd0195330472c88b240a7f2192830 P c70e07315fc54089ac8d880c6ea968ed: Deleting consensus metadata
I20260806 03:52:11.180351 24113 catalog_manager.cc:4841] TS c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723): tablet 454fd0195330472c88b240a7f2192830 (table test-table [id=92bd3b30b58348fdbdb201731804a366]) successfully deleted
I20260806 03:52:11.181602 24957 tablet_replica.cc:330] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: stopping tablet replica
I20260806 03:52:11.181663 24957 raft_consensus.cc:2229] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:52:11.181790 24957 raft_consensus.cc:2258] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.182022 24957 ts_tablet_manager.cc:1905] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:11.183540 24957 ts_tablet_manager.cc:1918] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:52:11.183604 24957 log.cc:1196] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/454fd0195330472c88b240a7f2192830
I20260806 03:52:11.183827 24957 ts_tablet_manager.cc:1939] T 454fd0195330472c88b240a7f2192830 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting consensus metadata
I20260806 03:52:11.184151 24116 catalog_manager.cc:4841] TS 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233): tablet 454fd0195330472c88b240a7f2192830 (table test-table [id=92bd3b30b58348fdbdb201731804a366]) successfully deleted
I20260806 03:52:11.184334 24956 tablet_replica.cc:330] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: stopping tablet replica
I20260806 03:52:11.184439 24956 raft_consensus.cc:2229] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:11.184525 24956 raft_consensus.cc:2258] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.184839 24956 ts_tablet_manager.cc:1905] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:11.186499 24956 ts_tablet_manager.cc:1918] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:52:11.186606 24956 log.cc:1196] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/454fd0195330472c88b240a7f2192830
I20260806 03:52:11.186857 24956 ts_tablet_manager.cc:1939] T 454fd0195330472c88b240a7f2192830 P 99453e44a16a4ea2bf6a72b07031e315: Deleting consensus metadata
I20260806 03:52:11.187175 24110 catalog_manager.cc:4841] TS 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223): tablet 454fd0195330472c88b240a7f2192830 (table test-table [id=92bd3b30b58348fdbdb201731804a366]) successfully deleted
Aug 06 03:52:11 dist-test-slave-bqcl krb5kdc[24067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 06 03:52:11 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:11 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:52:11.212423 24908 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:11.204650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55746 (local address 127.12.15.126:34289)
0806 03:52:11.204704 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.204706 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.204842 (+ 136us) server_negotiation.cc:407] Connection header received
0806 03:52:11.204924 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.204925 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.204938 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.205007 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:11.205745 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.206599 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.208515 (+ 1916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.208801 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.208872 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.212360 (+ 3488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55746: BlockingRecv error: recv got EOF from 127.0.0.1:55746 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Aug 06 03:52:11 dist-test-slave-bqcl krb5kdc[24067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988331, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM
I20260806 03:52:11.221613 24908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.213160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55750 (local address 127.12.15.126:34289)
0806 03:52:11.213195 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.213197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.213441 (+ 244us) server_negotiation.cc:407] Connection header received
0806 03:52:11.213504 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.213505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.213517 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.213574 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:11.214099 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.214863 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.215825 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.216125 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.216190 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.218748 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:11.218750 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:11.218750 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:11.218758 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:11.220374 (+ 1616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:11.220774 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:11.220775 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:11.220775 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:11.220875 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:11.221072 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:11.221072 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:11.221073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:11.221199 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:11.221296 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.221400 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.221485 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:52:11.223160 24136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:52:11.223335 24136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:52:11.224666 24332 tablet_service.cc:1467] Processing CreateTablet for tablet 85686c7a1e9b434b900e9efb624530d3 (DEFAULT_TABLE table=test-table [id=9d207176aec243c8a39aaa14337d19f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:11.224812 24332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85686c7a1e9b434b900e9efb624530d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:11.224838 24523 tablet_service.cc:1467] Processing CreateTablet for tablet 85686c7a1e9b434b900e9efb624530d3 (DEFAULT_TABLE table=test-table [id=9d207176aec243c8a39aaa14337d19f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:11.224962 24523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85686c7a1e9b434b900e9efb624530d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:11.226311 24720 tablet_service.cc:1467] Processing CreateTablet for tablet 85686c7a1e9b434b900e9efb624530d3 (DEFAULT_TABLE table=test-table [id=9d207176aec243c8a39aaa14337d19f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:11.226408 24851 tablet_bootstrap.cc:492] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: Bootstrap starting.
I20260806 03:52:11.226433 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85686c7a1e9b434b900e9efb624530d3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:11.226810 24851 tablet_bootstrap.cc:654] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:11.227672 24850 tablet_bootstrap.cc:492] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: Bootstrap starting.
I20260806 03:52:11.227679 24849 tablet_bootstrap.cc:492] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: Bootstrap starting.
I20260806 03:52:11.228019 24850 tablet_bootstrap.cc:654] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:11.228080 24849 tablet_bootstrap.cc:654] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:11.228600 24850 tablet_bootstrap.cc:492] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: No bootstrap required, opened a new log
I20260806 03:52:11.228652 24850 ts_tablet_manager.cc:1397] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260806 03:52:11.228787 24850 raft_consensus.cc:348] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.228852 24850 raft_consensus.cc:374] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:11.228869 24850 raft_consensus.cc:729] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70e07315fc54089ac8d880c6ea968ed, State: Initialized, Role: FOLLOWER
I20260806 03:52:11.228909 24850 consensus_queue.cc:260] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.228991 24850 ts_tablet_manager.cc:1428] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:11.229125 24849 tablet_bootstrap.cc:492] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: No bootstrap required, opened a new log
I20260806 03:52:11.229166 24849 ts_tablet_manager.cc:1397] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:11.229282 24849 raft_consensus.cc:348] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.229339 24849 raft_consensus.cc:374] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:11.229357 24849 raft_consensus.cc:729] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb6f5d5214c4003b5908367856c7ef2, State: Initialized, Role: FOLLOWER
I20260806 03:52:11.229395 24849 consensus_queue.cc:260] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.229470 24849 ts_tablet_manager.cc:1428] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:11.230293 24851 tablet_bootstrap.cc:492] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: No bootstrap required, opened a new log
I20260806 03:52:11.230340 24851 ts_tablet_manager.cc:1397] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.001s
I20260806 03:52:11.230473 24851 raft_consensus.cc:348] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.230545 24851 raft_consensus.cc:374] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:11.230566 24851 raft_consensus.cc:729] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99453e44a16a4ea2bf6a72b07031e315, State: Initialized, Role: FOLLOWER
I20260806 03:52:11.230599 24851 consensus_queue.cc:260] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.230688 24851 ts_tablet_manager.cc:1428] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:11.283535 24876 raft_consensus.cc:482] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:11.283655 24876 raft_consensus.cc:504] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.283850 24876 leader_election.cc:288] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223), 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233)
I20260806 03:52:11.284082 24359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85686c7a1e9b434b900e9efb624530d3" candidate_uuid: "c70e07315fc54089ac8d880c6ea968ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99453e44a16a4ea2bf6a72b07031e315" is_pre_election: true
I20260806 03:52:11.284082 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85686c7a1e9b434b900e9efb624530d3" candidate_uuid: "c70e07315fc54089ac8d880c6ea968ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb6f5d5214c4003b5908367856c7ef2" is_pre_election: true
I20260806 03:52:11.284195 24556 raft_consensus.cc:2454] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70e07315fc54089ac8d880c6ea968ed in term 0.
I20260806 03:52:11.284195 24359 raft_consensus.cc:2454] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c70e07315fc54089ac8d880c6ea968ed in term 0.
I20260806 03:52:11.284349 24653 leader_election.cc:302] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb6f5d5214c4003b5908367856c7ef2, c70e07315fc54089ac8d880c6ea968ed; no voters:
I20260806 03:52:11.284475 24876 raft_consensus.cc:2781] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:52:11.284538 24876 raft_consensus.cc:482] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:11.284557 24876 raft_consensus.cc:3037] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.285523 24876 raft_consensus.cc:504] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.285665 24876 leader_election.cc:288] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [CANDIDATE]: Term 1 election: Requested vote from peers 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223), 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233)
I20260806 03:52:11.285800 24359 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85686c7a1e9b434b900e9efb624530d3" candidate_uuid: "c70e07315fc54089ac8d880c6ea968ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "99453e44a16a4ea2bf6a72b07031e315"
I20260806 03:52:11.285817 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85686c7a1e9b434b900e9efb624530d3" candidate_uuid: "c70e07315fc54089ac8d880c6ea968ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb6f5d5214c4003b5908367856c7ef2"
I20260806 03:52:11.285876 24556 raft_consensus.cc:3037] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.285876 24359 raft_consensus.cc:3037] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.286612 24556 raft_consensus.cc:2454] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70e07315fc54089ac8d880c6ea968ed in term 1.
I20260806 03:52:11.286612 24359 raft_consensus.cc:2454] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c70e07315fc54089ac8d880c6ea968ed in term 1.
I20260806 03:52:11.286775 24650 leader_election.cc:302] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 99453e44a16a4ea2bf6a72b07031e315, c70e07315fc54089ac8d880c6ea968ed; no voters:
I20260806 03:52:11.286892 24876 raft_consensus.cc:2781] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:11.287071 24876 raft_consensus.cc:686] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 1 LEADER]: Becoming Leader. State: Replica: c70e07315fc54089ac8d880c6ea968ed, State: Running, Role: LEADER
I20260806 03:52:11.287143 24876 consensus_queue.cc:237] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.287694 24134 catalog_manager.cc:5495] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed reported cstate change: term changed from 0 to 1, leader changed from <none> to c70e07315fc54089ac8d880c6ea968ed (127.12.15.67). New cstate: current_term: 1 leader_uuid: "c70e07315fc54089ac8d880c6ea968ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } health_report { overall_health: HEALTHY } } }
I20260806 03:52:11.305830 24846 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.303736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44894 (local address 127.12.15.67:42723)
0806 03:52:11.303784 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.303786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.303808 (+ 22us) server_negotiation.cc:407] Connection header received
0806 03:52:11.303830 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.303830 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.303840 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.303889 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:11.303988 (+ 99us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.304464 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.305220 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.305424 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.305478 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.305610 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.305643 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.305683 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.305737 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.305759 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:52:11.306504 24556 raft_consensus.cc:1261] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Refusing update from remote peer c70e07315fc54089ac8d880c6ea968ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:11.306543 24359 raft_consensus.cc:1261] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Refusing update from remote peer c70e07315fc54089ac8d880c6ea968ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:11.306730 24978 consensus_queue.cc:1035] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:11.306845 24876 consensus_queue.cc:1035] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:11.309177 24136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:11.309265 24136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:11.309793 24136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: Sending DeleteTablet for 3 replicas of tablet 85686c7a1e9b434b900e9efb624530d3
I20260806 03:52:11.310065 24720 tablet_service.cc:1514] Processing DeleteTablet for tablet 85686c7a1e9b434b900e9efb624530d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:44872
I20260806 03:52:11.310182 24332 tablet_service.cc:1514] Processing DeleteTablet for tablet 85686c7a1e9b434b900e9efb624530d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:43352
I20260806 03:52:11.310281 24523 tablet_service.cc:1514] Processing DeleteTablet for tablet 85686c7a1e9b434b900e9efb624530d3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:47078
I20260806 03:52:11.310278 24136 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260806 03:52:11.310365 24958 tablet_replica.cc:330] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: stopping tablet replica
W20260806 03:52:11.310413 24136 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:52:11.310418 24958 raft_consensus.cc:2229] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:52:11.310469 24958 raft_consensus.cc:2258] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.310545 24956 tablet_replica.cc:330] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: stopping tablet replica
I20260806 03:52:11.310603 24956 raft_consensus.cc:2229] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:11.310645 24956 raft_consensus.cc:2258] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.310711 24957 tablet_replica.cc:330] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: stopping tablet replica
I20260806 03:52:11.310756 24957 raft_consensus.cc:2229] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:11.310789 24957 raft_consensus.cc:2258] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.311452 24957 ts_tablet_manager.cc:1905] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:11.311599 24958 ts_tablet_manager.cc:1905] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:11.313015 24958 ts_tablet_manager.cc:1918] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:11.313056 24956 ts_tablet_manager.cc:1905] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:11.313076 24958 log.cc:1196] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/85686c7a1e9b434b900e9efb624530d3
I20260806 03:52:11.313278 24958 ts_tablet_manager.cc:1939] T 85686c7a1e9b434b900e9efb624530d3 P c70e07315fc54089ac8d880c6ea968ed: Deleting consensus metadata
I20260806 03:52:11.313544 24113 catalog_manager.cc:4841] TS c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723): tablet 85686c7a1e9b434b900e9efb624530d3 (table test-table [id=9d207176aec243c8a39aaa14337d19f0]) successfully deleted
I20260806 03:52:11.313793 24523 tablet_service.cc:1467] Processing CreateTablet for tablet 5af5defc327e43cb997086ed40792a79 (DEFAULT_TABLE table=test-table [id=8e8a80309fe4453e8a487734e5fa8f38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:11.313923 24720 tablet_service.cc:1467] Processing CreateTablet for tablet 5af5defc327e43cb997086ed40792a79 (DEFAULT_TABLE table=test-table [id=8e8a80309fe4453e8a487734e5fa8f38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:11.314015 24720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5af5defc327e43cb997086ed40792a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:11.314345 24957 ts_tablet_manager.cc:1918] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:11.314404 24957 log.cc:1196] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/85686c7a1e9b434b900e9efb624530d3
I20260806 03:52:11.314533 24956 ts_tablet_manager.cc:1918] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:11.314592 24957 ts_tablet_manager.cc:1939] T 85686c7a1e9b434b900e9efb624530d3 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting consensus metadata
I20260806 03:52:11.314597 24956 log.cc:1196] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/85686c7a1e9b434b900e9efb624530d3
I20260806 03:52:11.314798 24956 ts_tablet_manager.cc:1939] T 85686c7a1e9b434b900e9efb624530d3 P 99453e44a16a4ea2bf6a72b07031e315: Deleting consensus metadata
I20260806 03:52:11.315057 24110 catalog_manager.cc:4841] TS 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223): tablet 85686c7a1e9b434b900e9efb624530d3 (table test-table [id=9d207176aec243c8a39aaa14337d19f0]) successfully deleted
I20260806 03:52:11.315150 24332 tablet_service.cc:1467] Processing CreateTablet for tablet 5af5defc327e43cb997086ed40792a79 (DEFAULT_TABLE table=test-table [id=8e8a80309fe4453e8a487734e5fa8f38]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:11.315166 24850 tablet_bootstrap.cc:492] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: Bootstrap starting.
I20260806 03:52:11.315249 24332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5af5defc327e43cb997086ed40792a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:11.315399 24850 tablet_bootstrap.cc:654] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:11.315589 24523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5af5defc327e43cb997086ed40792a79. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:11.315814 24116 catalog_manager.cc:4841] TS 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233): tablet 85686c7a1e9b434b900e9efb624530d3 (table test-table [id=9d207176aec243c8a39aaa14337d19f0]) successfully deleted
I20260806 03:52:11.316176 24851 tablet_bootstrap.cc:492] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315: Bootstrap starting.
I20260806 03:52:11.316463 24851 tablet_bootstrap.cc:654] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:11.316646 24849 tablet_bootstrap.cc:492] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: Bootstrap starting.
I20260806 03:52:11.316783 24850 tablet_bootstrap.cc:492] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: No bootstrap required, opened a new log
I20260806 03:52:11.316828 24850 ts_tablet_manager.cc:1397] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.000s
I20260806 03:52:11.316933 24849 tablet_bootstrap.cc:654] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:11.316915 24850 raft_consensus.cc:348] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.316967 24850 raft_consensus.cc:374] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:11.316982 24850 raft_consensus.cc:729] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70e07315fc54089ac8d880c6ea968ed, State: Initialized, Role: FOLLOWER
I20260806 03:52:11.317014 24850 consensus_queue.cc:260] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.317098 24850 ts_tablet_manager.cc:1428] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:11.317428 24851 tablet_bootstrap.cc:492] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315: No bootstrap required, opened a new log
I20260806 03:52:11.317476 24851 ts_tablet_manager.cc:1397] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260806 03:52:11.317586 24851 raft_consensus.cc:348] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.317659 24851 raft_consensus.cc:374] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:11.317673 24851 raft_consensus.cc:729] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 99453e44a16a4ea2bf6a72b07031e315, State: Initialized, Role: FOLLOWER
I20260806 03:52:11.317718 24851 consensus_queue.cc:260] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.317809 24851 ts_tablet_manager.cc:1428] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:11.318024 24849 tablet_bootstrap.cc:492] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: No bootstrap required, opened a new log
I20260806 03:52:11.318074 24849 ts_tablet_manager.cc:1397] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:11.318203 24849 raft_consensus.cc:348] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.318274 24849 raft_consensus.cc:374] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:11.318295 24849 raft_consensus.cc:729] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fb6f5d5214c4003b5908367856c7ef2, State: Initialized, Role: FOLLOWER
I20260806 03:52:11.318333 24849 consensus_queue.cc:260] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.318413 24849 ts_tablet_manager.cc:1428] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:11.519304 24918 raft_consensus.cc:482] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:11.519420 24918 raft_consensus.cc:504] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.519587 24918 leader_election.cc:288] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233), c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723)
I20260806 03:52:11.519811 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5af5defc327e43cb997086ed40792a79" candidate_uuid: "99453e44a16a4ea2bf6a72b07031e315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb6f5d5214c4003b5908367856c7ef2" is_pre_election: true
I20260806 03:52:11.519838 24750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5af5defc327e43cb997086ed40792a79" candidate_uuid: "99453e44a16a4ea2bf6a72b07031e315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70e07315fc54089ac8d880c6ea968ed" is_pre_election: true
I20260806 03:52:11.519912 24556 raft_consensus.cc:2454] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99453e44a16a4ea2bf6a72b07031e315 in term 0.
I20260806 03:52:11.519922 24750 raft_consensus.cc:2454] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 99453e44a16a4ea2bf6a72b07031e315 in term 0.
I20260806 03:52:11.520072 24265 leader_election.cc:302] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb6f5d5214c4003b5908367856c7ef2, 99453e44a16a4ea2bf6a72b07031e315; no voters:
I20260806 03:52:11.520182 24918 raft_consensus.cc:2781] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:52:11.520241 24918 raft_consensus.cc:482] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:11.520268 24918 raft_consensus.cc:3037] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.521018 24918 raft_consensus.cc:504] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.521142 24918 leader_election.cc:288] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [CANDIDATE]: Term 1 election: Requested vote from peers 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233), c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723)
I20260806 03:52:11.521291 24556 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5af5defc327e43cb997086ed40792a79" candidate_uuid: "99453e44a16a4ea2bf6a72b07031e315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6fb6f5d5214c4003b5908367856c7ef2"
I20260806 03:52:11.521315 24750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5af5defc327e43cb997086ed40792a79" candidate_uuid: "99453e44a16a4ea2bf6a72b07031e315" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c70e07315fc54089ac8d880c6ea968ed"
I20260806 03:52:11.521356 24556 raft_consensus.cc:3037] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.521384 24750 raft_consensus.cc:3037] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:11.521952 24556 raft_consensus.cc:2454] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99453e44a16a4ea2bf6a72b07031e315 in term 1.
I20260806 03:52:11.522023 24750 raft_consensus.cc:2454] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 99453e44a16a4ea2bf6a72b07031e315 in term 1.
I20260806 03:52:11.522151 24265 leader_election.cc:302] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6fb6f5d5214c4003b5908367856c7ef2, 99453e44a16a4ea2bf6a72b07031e315; no voters:
I20260806 03:52:11.522235 24918 raft_consensus.cc:2781] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:11.522289 24918 raft_consensus.cc:686] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 LEADER]: Becoming Leader. State: Replica: 99453e44a16a4ea2bf6a72b07031e315, State: Running, Role: LEADER
I20260806 03:52:11.522329 24918 consensus_queue.cc:237] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } }
I20260806 03:52:11.522868 24136 catalog_manager.cc:5495] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 reported cstate change: term changed from 0 to 1, leader changed from <none> to 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "99453e44a16a4ea2bf6a72b07031e315" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "99453e44a16a4ea2bf6a72b07031e315" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44223 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 } health_report { overall_health: UNKNOWN } } }
I20260806 03:52:11.543047 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.540925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43378 (local address 127.12.15.65:44223)
0806 03:52:11.540968 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.540970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.541002 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:52:11.541065 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.541066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.541075 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.541126 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:11.541222 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.541688 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.542457 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.542648 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.542702 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.542827 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.542860 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.542899 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.542951 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.542974 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:52:11.546763 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.544997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:43437 (local address 127.12.15.65:44223)
0806 03:52:11.545033 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:11.545035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:11.545042 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:11.545062 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:11.545063 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:11.545070 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:11.545114 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:11.545189 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.545650 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.546298 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.546467 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.546535 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.546591 (+ 56us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.546613 (+ 22us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.546645 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:11.546684 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 03:52:11.546696 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
I20260806 03:52:11.546819 24950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.544930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44223 (local address 127.12.15.65:43437)
0806 03:52:11.544977 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:11.544986 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:11.545016 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:11.545121 (+ 105us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:11.545122 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:11.545122 (+ 0us) client_negotiation.cc:189] Negotiated authn=TOKEN
0806 03:52:11.545154 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.545155 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.545700 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.545701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.546251 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:11.546252 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:11.546519 (+ 267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:11.546520 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:11.546552 (+ 32us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:11.546556 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0806 03:52:11.546652 (+ 96us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0806 03:52:11.546653 (+ 1us) client_negotiation.cc:769] Sending connection context
0806 03:52:11.546682 (+ 29us) client_negotiation.cc:240] Negotiation successful
0806 03:52:11.546722 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 03:52:11.548552 24750 raft_consensus.cc:1261] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Refusing update from remote peer 99453e44a16a4ea2bf6a72b07031e315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:11.548558 24556 raft_consensus.cc:1261] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Refusing update from remote peer 99453e44a16a4ea2bf6a72b07031e315: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:11.548732 24918 consensus_queue.cc:1035] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6fb6f5d5214c4003b5908367856c7ef2" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45233 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:11.548853 24932 consensus_queue.cc:1035] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c70e07315fc54089ac8d880c6ea968ed" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42723 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:11.558365 24136 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:11.558456 24136 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:11.559046 24136 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 959beb2c2d5b417daf47b27c92bbbd5f: Sending DeleteTablet for 3 replicas of tablet 5af5defc327e43cb997086ed40792a79
I20260806 03:52:11.559302 24523 tablet_service.cc:1514] Processing DeleteTablet for tablet 5af5defc327e43cb997086ed40792a79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:47078
I20260806 03:52:11.559345 24720 tablet_service.cc:1514] Processing DeleteTablet for tablet 5af5defc327e43cb997086ed40792a79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:44872
I20260806 03:52:11.559398 24957 tablet_replica.cc:330] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: stopping tablet replica
I20260806 03:52:11.559444 24957 raft_consensus.cc:2229] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:11.559434 24332 tablet_service.cc:1514] Processing DeleteTablet for tablet 5af5defc327e43cb997086ed40792a79 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:11 UTC) from {username='oryx', principal='oryx/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:43352
I20260806 03:52:11.559478 24957 raft_consensus.cc:2258] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.559505 24958 tablet_replica.cc:330] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: stopping tablet replica
I20260806 03:52:11.559556 24958 raft_consensus.cc:2229] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:11.559592 24958 raft_consensus.cc:2258] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.559669 24956 tablet_replica.cc:330] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315: stopping tablet replica
I20260806 03:52:11.559731 24956 raft_consensus.cc:2229] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:52:11.559821 24956 raft_consensus.cc:2258] T 5af5defc327e43cb997086ed40792a79 P 99453e44a16a4ea2bf6a72b07031e315 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:11.559844 24957 ts_tablet_manager.cc:1905] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:11.560129 24958 ts_tablet_manager.cc:1905] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:11.561146 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 24239
I20260806 03:52:11.563671 24958 ts_tablet_manager.cc:1918] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:52:11.563731 24958 log.cc:1196] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/5af5defc327e43cb997086ed40792a79
I20260806 03:52:11.563941 24958 ts_tablet_manager.cc:1939] T 5af5defc327e43cb997086ed40792a79 P c70e07315fc54089ac8d880c6ea968ed: Deleting consensus metadata
I20260806 03:52:11.564266 24113 catalog_manager.cc:4841] TS c70e07315fc54089ac8d880c6ea968ed (127.12.15.67:42723): tablet 5af5defc327e43cb997086ed40792a79 (table test-table [id=8e8a80309fe4453e8a487734e5fa8f38]) successfully deleted
I20260806 03:52:11.567847 24957 ts_tablet_manager.cc:1918] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260806 03:52:11.567909 24957 log.cc:1196] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/5af5defc327e43cb997086ed40792a79
I20260806 03:52:11.568092 24957 ts_tablet_manager.cc:1939] T 5af5defc327e43cb997086ed40792a79 P 6fb6f5d5214c4003b5908367856c7ef2: Deleting consensus metadata
I20260806 03:52:11.568374 24116 catalog_manager.cc:4841] TS 6fb6f5d5214c4003b5908367856c7ef2 (127.12.15.66:45233): tablet 5af5defc327e43cb997086ed40792a79 (table test-table [id=8e8a80309fe4453e8a487734e5fa8f38]) successfully deleted
W20260806 03:52:11.574599 24110 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:44223 (error 108)
I20260806 03:52:11.574815 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 24434
I20260806 03:52:11.581395 24999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:11.576965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44223 (local address 127.0.0.1:43382)
0806 03:52:11.581231 (+ 4266us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:11.581331 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:44223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4235,"thread_start_us":1353,"threads_started":1}
W20260806 03:52:11.581619 24110 catalog_manager.cc:4584] TS 99453e44a16a4ea2bf6a72b07031e315 (127.12.15.65:44223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5af5defc327e43cb997086ed40792a79: Network error: Client connection negotiation failed: client connection to 127.12.15.65:44223: connect: Connection refused (error 111)
I20260806 03:52:11.589174 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 24629
I20260806 03:52:11.599418 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 24090
2026-08-06T03:52:11Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3096 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:52:11 dist-test-slave-bqcl krb5kdc[25003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:52:11 dist-test-slave-bqcl krb5kdc[25003](info): set up 2 sockets
Aug 06 03:52:11 dist-test-slave-bqcl krb5kdc[25003](info): commencing operation
krb5kdc: starting...
W20260806 03:52:12.736712 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:52:12 dist-test-slave-bqcl krb5kdc[25003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988332, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:52:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:52:12.856294 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:34431
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:37205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:34431
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:12.936695 25021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:12.936902 25021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:12.936932 25021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:12.938048 25021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:12.938090 25021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:12.938108 25021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:12.938151 25021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:12.938169 25021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:12.939639 25021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:37205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:34431
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:34431
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:12.939881 25021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:12.940092 25021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:12.944737 25030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:12.944792 25040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:12.944871 25035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:12.944985 25021 server_base.cc:1034] running on GCE node
I20260806 03:52:12.945245 25021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:12.945660 25021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:12.946811 25021 hybrid_clock.cc:648] HybridClock initialized: now 1785988332946788 us; error 37 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-06T03:52:12Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1349 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:52:13 dist-test-slave-bqcl krb5kdc[25048](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:52:13 dist-test-slave-bqcl krb5kdc[25048](info): set up 2 sockets
Aug 06 03:52:13 dist-test-slave-bqcl krb5kdc[25048](info): commencing operation
krb5kdc: starting...
W20260806 03:52:14.092031 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:52:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:52:14.209167 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:42617
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:42617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:14.289657 25065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:14.289846 25065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:14.289866 25065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:14.290951 25065 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:14.290987 25065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:14.290999 25065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:14.291010 25065 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:14.291020 25065 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:14.292400 25065 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:42617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25065
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:14.292600 25065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:14.292773 25065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:14.297335 25074 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:14.297410 25070 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:14.297343 25072 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:14.297638 25065 server_base.cc:1034] running on GCE node
I20260806 03:52:14.297928 25065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:14.298252 25065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:14.299410 25065 hybrid_clock.cc:648] HybridClock initialized: now 1785988334299393 us; error 28 us; skew 500 ppm
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:14.305644 25065 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:52:14.306102 25065 webserver.cc:460] Webserver started at http://127.12.15.126:38005/ using document root <none> and password file <none>
I20260806 03:52:14.306354 25065 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:14.306399 25065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:14.306494 25065 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:14.307487 25065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "38a46ac9621946259f16e09a816efdf3"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "7655226a1bc90d88842538d357a739dc"
server_key_iv: "e0b2defbef8a6676d4d9cd3e8fe8fc2f"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.307797 25065 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "38a46ac9621946259f16e09a816efdf3"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "7655226a1bc90d88842538d357a739dc"
server_key_iv: "e0b2defbef8a6676d4d9cd3e8fe8fc2f"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.308976 25065 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:14.309631 25085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.309799 25065 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:14.309885 25065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "38a46ac9621946259f16e09a816efdf3"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "7655226a1bc90d88842538d357a739dc"
server_key_iv: "e0b2defbef8a6676d4d9cd3e8fe8fc2f"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.309942 25065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:14.332785 25065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:14.334592 25065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:14.334702 25065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:14.338382 25065 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:42617
I20260806 03:52:14.338425 25161 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:42617 every 8 connection(s)
I20260806 03:52:14.338744 25065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:52:14.339613 25162 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:14.341797 25162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Bootstrap starting.
I20260806 03:52:14.342391 25162 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:14.342659 25162 log.cc:824] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:14.343318 25162 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: No bootstrap required, opened a new log
I20260806 03:52:14.343822 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 25065
I20260806 03:52:14.343938 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:52:14.344129 12349 external_mini_cluster.cc:1428] Setting key 5c7f084031e327a2ae0f12f97d8d13f6
I20260806 03:52:14.345355 25162 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } }
I20260806 03:52:14.345500 25162 raft_consensus.cc:374] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:14.345536 25162 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a46ac9621946259f16e09a816efdf3, State: Initialized, Role: FOLLOWER
I20260806 03:52:14.345659 25162 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } }
I20260806 03:52:14.345773 25162 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:52:14.345813 25162 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:52:14.345861 25162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:14.346794 25162 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } }
I20260806 03:52:14.346884 25162 leader_election.cc:302] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38a46ac9621946259f16e09a816efdf3; no voters:
I20260806 03:52:14.347031 25162 leader_election.cc:288] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:52:14.347090 25167 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:14.347223 25167 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 1 LEADER]: Becoming Leader. State: Replica: 38a46ac9621946259f16e09a816efdf3, State: Running, Role: LEADER
I20260806 03:52:14.347290 25162 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:14.347306 25167 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } }
I20260806 03:52:14.347725 25168 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "38a46ac9621946259f16e09a816efdf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } } }
I20260806 03:52:14.347810 25168 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:14.348021 25169 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38a46ac9621946259f16e09a816efdf3. Latest consensus state: current_term: 1 leader_uuid: "38a46ac9621946259f16e09a816efdf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } } }
I20260806 03:52:14.348129 25169 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:14.348359 25178 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:14.348830 25178 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:14.350807 25178 catalog_manager.cc:1347] Generated new cluster ID: 25b1e3cefbdf42898bd2dc873c30142e
I20260806 03:52:14.350862 25178 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:14.355451 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:14.344633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46030 (local address 127.12.15.126:42617)
0806 03:52:14.344800 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:14.344809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:14.344835 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:52:14.345320 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:14.345325 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:14.345370 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:14.345477 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:14.346284 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.347352 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.348418 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.349023 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.349308 (+ 285us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:14.351864 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:14.351869 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:14.351869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:14.351881 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:14.353795 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:14.354207 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:14.354211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:14.354211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:14.354358 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:14.354559 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:14.354559 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:14.354560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:14.354861 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:14.355027 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:14.355108 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 03:52:14.355253 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":78,"threads_started":1}
I20260806 03:52:14.373122 25178 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:52:14.373683 25178 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:52:14.379115 25178 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Generated new TSK 0
I20260806 03:52:14.379295 25178 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:52:14.413364 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42617
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:14.493269 25199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:14.493471 25199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:14.493494 25199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:14.494659 25199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:14.494700 25199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:14.494724 25199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:14.496121 25199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:14.496340 25199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:14.496551 25199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:14.501235 25207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:14.501263 25210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:14.501327 25208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:14.501500 25199 server_base.cc:1034] running on GCE node
I20260806 03:52:14.501703 25199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:14.501921 25199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:14.503062 25199 hybrid_clock.cc:648] HybridClock initialized: now 1785988334503050 us; error 30 us; skew 500 ppm
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:14.505414 25199 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:52:14.505824 25199 webserver.cc:460] Webserver started at http://127.12.15.65:34153/ using document root <none> and password file <none>
I20260806 03:52:14.506013 25199 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:14.506058 25199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:14.506217 25199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:14.507174 25199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "ef08e8157ba04003a44a480009d1d20b"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "c0cd371efb883b49fb5217cfa4cede8e"
server_key_iv: "e9b45f468418951554017fbded6b3271"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.507500 25199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "ef08e8157ba04003a44a480009d1d20b"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "c0cd371efb883b49fb5217cfa4cede8e"
server_key_iv: "e9b45f468418951554017fbded6b3271"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.508731 25199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:52:14.509338 25221 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.509496 25199 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:14.509578 25199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "ef08e8157ba04003a44a480009d1d20b"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "c0cd371efb883b49fb5217cfa4cede8e"
server_key_iv: "e9b45f468418951554017fbded6b3271"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.509641 25199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:14.516731 25199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:14.518383 25199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:14.518509 25199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:14.518700 25199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:14.518982 25199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:14.519016 25199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.519042 25199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:14.519059 25199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.524394 25199 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:41573
I20260806 03:52:14.524452 25381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:41573 every 8 connection(s)
I20260806 03:52:14.524720 25199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:52:14.528478 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 25199
I20260806 03:52:14.528602 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:52:14.528784 12349 external_mini_cluster.cc:1428] Setting key eae71d34d1a21163d1783de58ee4f4a4
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:14.533814 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:14.525371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:43255 (local address 127.12.15.126:42617)
0806 03:52:14.525442 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:14.525445 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:14.525643 (+ 198us) server_negotiation.cc:407] Connection header received
0806 03:52:14.526081 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:14.526082 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:14.526092 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:14.526164 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:14.527050 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.527545 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.528536 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.528810 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.528871 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:14.531154 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:14.531156 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:14.531156 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:14.531165 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:14.532567 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:14.532935 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:14.532937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:14.532937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:14.533005 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:14.533208 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:14.533208 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:14.533209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:14.533334 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:14.533397 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:14.533619 (+ 222us) server_negotiation.cc:299] Negotiation successful
0806 03:52:14.533723 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:52:14.533843 25386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:14.525338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:43255)
0806 03:52:14.525496 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:14.525530 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:14.526001 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:14.526240 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:14.526243 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:14.526800 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:14.526979 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:14.526984 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.527609 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.527610 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:14.528462 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:14.528465 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.528936 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.528937 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:14.529042 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:14.529508 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:14.529516 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:14.531049 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:14.532651 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:14.532657 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:14.532659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:14.532866 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:14.533065 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:14.533065 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:14.533065 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:14.533150 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:14.533388 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:14.533390 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:14.533493 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 03:52:14.533571 (+ 78us) client_negotiation.cc:240] Negotiation successful
0806 03:52:14.533689 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":87,"threads_started":1}
I20260806 03:52:14.534332 25382 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42617
I20260806 03:52:14.534433 25382 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:14.535039 25382 heartbeater.cc:511] Master 127.12.15.126:42617 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
I20260806 03:52:14.541136 25106 ts_manager.cc:194] Registered new tserver with Master: ef08e8157ba04003a44a480009d1d20b (127.12.15.65:41573)
I20260806 03:52:14.541990 25106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:43255
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:52:14.554864 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42617
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:14.635202 25391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:14.635404 25391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:14.635425 25391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:14.636552 25391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:14.636595 25391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:14.636633 25391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:14.638020 25391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:14.638252 25391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:14.638464 25391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:14.643265 25397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:14.643253 25401 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:14.643424 25391 server_base.cc:1034] running on GCE node
W20260806 03:52:14.643301 25398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:14.643658 25391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:14.643905 25391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:14.645045 25391 hybrid_clock.cc:648] HybridClock initialized: now 1785988334645028 us; error 29 us; skew 500 ppm
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:14.647431 25391 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:52:14.647863 25391 webserver.cc:460] Webserver started at http://127.12.15.66:43933/ using document root <none> and password file <none>
I20260806 03:52:14.648061 25391 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:14.648109 25391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:14.648231 25391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:14.649224 25391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "f0b91a8ecb6a46698589241aef39595f"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "138042fbfbb206a66e9126f7cbeed85e"
server_key_iv: "28fa525eedd0b008d2e01f09d4264228"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.649546 25391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "f0b91a8ecb6a46698589241aef39595f"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "138042fbfbb206a66e9126f7cbeed85e"
server_key_iv: "28fa525eedd0b008d2e01f09d4264228"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.650916 25391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260806 03:52:14.651578 25409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.651747 25391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:14.651841 25391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "f0b91a8ecb6a46698589241aef39595f"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "138042fbfbb206a66e9126f7cbeed85e"
server_key_iv: "28fa525eedd0b008d2e01f09d4264228"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.651902 25391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:14.667943 25391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:14.669584 25391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:14.669724 25391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:14.669924 25391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:14.670285 25391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:14.670324 25391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.670353 25391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:14.670377 25391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.675792 25391 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:43157
I20260806 03:52:14.675832 25593 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:43157 every 8 connection(s)
I20260806 03:52:14.676121 25391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:52:14.679304 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 25391
I20260806 03:52:14.679399 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:52:14.679569 12349 external_mini_cluster.cc:1428] Setting key 39aa68d1d1982c8c44bb0cdde1c4f274
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:14.684770 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:14.676778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:54061 (local address 127.12.15.126:42617)
0806 03:52:14.676864 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:14.676867 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:14.677021 (+ 154us) server_negotiation.cc:407] Connection header received
0806 03:52:14.677466 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:14.677467 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:14.677477 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:14.677526 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:14.678392 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.678880 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.679933 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.680219 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.680274 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:14.682285 (+ 2011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:14.682286 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:14.682287 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:14.682294 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:14.683513 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:14.683884 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:14.683886 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:14.683886 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:14.683940 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:14.684163 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:14.684164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:14.684164 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:14.684275 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:14.684332 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:14.684587 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 03:52:14.684671 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260806 03:52:14.684909 25600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:14.676746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:54061)
0806 03:52:14.676905 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:14.676941 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:14.677390 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:14.677579 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:14.677582 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:14.678142 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:14.678327 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:14.678332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.678944 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.678944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:14.679857 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:14.679860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.680297 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.680298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:14.680409 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:14.680857 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:14.680865 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:14.682199 (+ 1334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:14.683591 (+ 1392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:14.683597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:14.683599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:14.683815 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:14.683998 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:14.683999 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:14.683999 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:14.684106 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:14.684334 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:14.684341 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:14.684469 (+ 128us) client_negotiation.cc:769] Sending connection context
0806 03:52:14.684555 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 03:52:14.684704 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260806 03:52:14.685560 25597 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42617
I20260806 03:52:14.685644 25597 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:14.685810 25597 heartbeater.cc:511] Master 127.12.15.126:42617 requested a full tablet report, sending...
I20260806 03:52:14.686203 25106 ts_manager.cc:194] Registered new tserver with Master: f0b91a8ecb6a46698589241aef39595f (127.12.15.66:43157)
I20260806 03:52:14.686704 25106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:54061
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:52:14.704984 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42617
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:14.784197 25605 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:14.784392 25605 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:14.784415 25605 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:14.785542 25605 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:14.785581 25605 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:14.785609 25605 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:14.787019 25605 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25605
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:14.787245 25605 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:14.787449 25605 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:14.792165 25610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:14.792191 25612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:14.792167 25614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:14.792426 25605 server_base.cc:1034] running on GCE node
I20260806 03:52:14.792614 25605 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:14.792833 25605 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:14.793989 25605 hybrid_clock.cc:648] HybridClock initialized: now 1785988334793979 us; error 30 us; skew 500 ppm
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:14.796398 25605 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:52:14.796811 25605 webserver.cc:460] Webserver started at http://127.12.15.67:45193/ using document root <none> and password file <none>
I20260806 03:52:14.796986 25605 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:14.797024 25605 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:14.797115 25605 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:14.798195 25605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "c6c8e1a72ed34f58adda940d1e91e1e4"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "9b73c0cdcec19d6ea7c5568748474b24"
server_key_iv: "ebafa87a21ebc86bbf28fd63cc3e23f9"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.798548 25605 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "c6c8e1a72ed34f58adda940d1e91e1e4"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "9b73c0cdcec19d6ea7c5568748474b24"
server_key_iv: "ebafa87a21ebc86bbf28fd63cc3e23f9"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.799731 25605 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:52:14.800343 25623 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.800493 25605 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:14.800573 25605 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "c6c8e1a72ed34f58adda940d1e91e1e4"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "9b73c0cdcec19d6ea7c5568748474b24"
server_key_iv: "ebafa87a21ebc86bbf28fd63cc3e23f9"
server_key_version: "encryptionkey@0"
I20260806 03:52:14.800633 25605 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:14.816960 25605 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:14.818984 25605 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:14.819115 25605 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:14.819329 25605 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:14.819612 25605 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:14.819645 25605 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.819672 25605 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:14.819698 25605 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:14.825136 25605 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:41555
I20260806 03:52:14.825205 25791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:41555 every 8 connection(s)
I20260806 03:52:14.825484 25605 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:52:14.829869 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 25605
I20260806 03:52:14.829990 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:52:14.830219 12349 external_mini_cluster.cc:1428] Setting key b159eae7e4ebb7448def7cad626d610e
Aug 06 03:52:14 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988334, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:14.833879 25166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:14.826168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:57731 (local address 127.12.15.126:42617)
0806 03:52:14.826217 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:14.826220 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:14.826389 (+ 169us) server_negotiation.cc:407] Connection header received
0806 03:52:14.826838 (+ 449us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:14.826839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:14.826849 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:14.826898 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:14.827755 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.828247 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.829158 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.829331 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.829371 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:14.831641 (+ 2270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:14.831642 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:14.831642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:14.831649 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:14.832894 (+ 1245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:14.833211 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:14.833211 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:14.833211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:14.833255 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:14.833426 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:14.833426 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:14.833427 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:14.833517 (+ 90us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:14.833560 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:14.833755 (+ 195us) server_negotiation.cc:299] Negotiation successful
0806 03:52:14.833816 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:52:14.834059 25795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:14.826105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:57731)
0806 03:52:14.826277 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:14.826310 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:14.826763 (+ 453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:14.826954 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:14.826957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:14.827495 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:14.827688 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:14.827693 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.828309 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.828310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:14.829099 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:14.829101 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:14.829385 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:14.829385 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:14.829476 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:14.829896 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:14.829906 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:14.831559 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:14.832953 (+ 1394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:14.832955 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:14.832957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:14.833147 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:14.833300 (+ 153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:14.833300 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:14.833301 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:14.833385 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:14.833558 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:14.833561 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:14.833667 (+ 106us) client_negotiation.cc:769] Sending connection context
0806 03:52:14.833742 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:52:14.833855 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":84,"threads_started":1}
I20260806 03:52:14.834465 25792 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42617
I20260806 03:52:14.834549 25792 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:14.834717 25792 heartbeater.cc:511] Master 127.12.15.126:42617 requested a full tablet report, sending...
I20260806 03:52:14.835047 25106 ts_manager.cc:194] Registered new tserver with Master: c6c8e1a72ed34f58adda940d1e91e1e4 (127.12.15.67:41555)
I20260806 03:52:14.835409 25106 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:57731
I20260806 03:52:14.841843 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab.
Entry for principal oryx/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab.
WARNING: no policy specified for oryx/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab.
Entry for principal oryx/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab.
WARNING: no policy specified for oryx/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab.
Entry for principal oryx/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab.
I20260806 03:52:14.908358 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 25199
I20260806 03:52:14.918762 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 25391
I20260806 03:52:14.933290 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 25605
I20260806 03:52:14.942955 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 25065
I20260806 03:52:14.951256 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:42617
--webserver_interface=127.12.15.126
--webserver_port=38005
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:42617
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:15.036553 25808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:15.036747 25808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:15.036777 25808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:15.037861 25808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:15.037901 25808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:15.037919 25808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:15.037935 25808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:15.037952 25808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:15.039463 25808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:42617
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:42617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=38005
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:15.039713 25808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:15.039917 25808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:15.044462 25813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:15.044414 25819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:15.044410 25815 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:15.044708 25808 server_base.cc:1034] running on GCE node
I20260806 03:52:15.044924 25808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:15.045172 25808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:15.046326 25808 hybrid_clock.cc:648] HybridClock initialized: now 1785988335046311 us; error 31 us; skew 500 ppm
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988335, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:15.048676 25808 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:52:15.049070 25808 webserver.cc:460] Webserver started at http://127.12.15.126:38005/ using document root <none> and password file <none>
I20260806 03:52:15.049275 25808 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:15.049336 25808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:15.050918 25808 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:15.051497 25829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.051671 25808 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:15.051756 25808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "38a46ac9621946259f16e09a816efdf3"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "7655226a1bc90d88842538d357a739dc"
server_key_iv: "e0b2defbef8a6676d4d9cd3e8fe8fc2f"
server_key_version: "encryptionkey@0"
I20260806 03:52:15.052076 25808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:15.063805 25808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:15.065431 25808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:15.065557 25808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:15.069270 25808 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:42617
I20260806 03:52:15.069317 25911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:42617 every 8 connection(s)
I20260806 03:52:15.069650 25808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:52:15.071110 25912 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:52:15.071707 25912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Bootstrap starting.
I20260806 03:52:15.075980 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 25808
I20260806 03:52:15.076246 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:41573
--local_ip_for_outbound_sockets=127.12.15.65
--tserver_master_addrs=127.12.15.126:42617
--webserver_port=34153
--webserver_interface=127.12.15.65
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.15.65
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:52:15.079838 25912 log.cc:824] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:15.081480 25912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:52:15.081729 25912 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Bootstrap complete.
I20260806 03:52:15.083794 25912 raft_consensus.cc:348] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } }
I20260806 03:52:15.084012 25912 raft_consensus.cc:729] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38a46ac9621946259f16e09a816efdf3, State: Initialized, Role: FOLLOWER
I20260806 03:52:15.084165 25912 consensus_queue.cc:260] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } }
I20260806 03:52:15.084246 25912 raft_consensus.cc:388] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:52:15.084297 25912 raft_consensus.cc:482] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:52:15.084345 25912 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:52:15.085314 25912 raft_consensus.cc:504] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } }
I20260806 03:52:15.085407 25912 leader_election.cc:302] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 38a46ac9621946259f16e09a816efdf3; no voters:
I20260806 03:52:15.085564 25912 leader_election.cc:288] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:52:15.085625 25918 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:52:15.085747 25918 raft_consensus.cc:686] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [term 2 LEADER]: Becoming Leader. State: Replica: 38a46ac9621946259f16e09a816efdf3, State: Running, Role: LEADER
I20260806 03:52:15.085835 25912 sys_catalog.cc:564] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:15.085832 25918 consensus_queue.cc:237] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } }
I20260806 03:52:15.086205 25919 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "38a46ac9621946259f16e09a816efdf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } } }
I20260806 03:52:15.086212 25918 sys_catalog.cc:455] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 38a46ac9621946259f16e09a816efdf3. Latest consensus state: current_term: 2 leader_uuid: "38a46ac9621946259f16e09a816efdf3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "38a46ac9621946259f16e09a816efdf3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42617 } } }
I20260806 03:52:15.086375 25919 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:15.086393 25918 sys_catalog.cc:458] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:15.086701 25929 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:15.087184 25929 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:15.087339 25929 catalog_manager.cc:1259] Loaded cluster ID: 25b1e3cefbdf42898bd2dc873c30142e
I20260806 03:52:15.087383 25929 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:52:15.088879 25929 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:52:15.089053 25929 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 38a46ac9621946259f16e09a816efdf3: Loaded TSK: 0
I20260806 03:52:15.089272 25929 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:52:15.159096 25916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:15.159312 25916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:15.159333 25916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:15.160490 25916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:15.160531 25916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:15.160557 25916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:15.162019 25916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.65.keytab
--principal=oryx/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:41573
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=34153
--tserver_master_addrs=127.12.15.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:15.162263 25916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:15.162488 25916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:15.168228 25946 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:15.168254 25945 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:15.168255 25951 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:15.168646 25916 server_base.cc:1034] running on GCE node
I20260806 03:52:15.168843 25916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:15.169057 25916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:15.170178 25916 hybrid_clock.cc:648] HybridClock initialized: now 1785988335170161 us; error 32 us; skew 500 ppm
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988335, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:15.172683 25916 init.cc:377] Logged in from keytab as oryx/127.12.15.65@KRBTEST.COM (short username oryx)
I20260806 03:52:15.173115 25916 webserver.cc:460] Webserver started at http://127.12.15.65:34153/ using document root <none> and password file <none>
I20260806 03:52:15.173329 25916 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:15.173384 25916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:15.174739 25916 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:52:15.175299 25963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.175464 25916 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:15.175567 25916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "ef08e8157ba04003a44a480009d1d20b"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "c0cd371efb883b49fb5217cfa4cede8e"
server_key_iv: "e9b45f468418951554017fbded6b3271"
server_key_version: "encryptionkey@0"
I20260806 03:52:15.176002 25916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:15.186740 25916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:15.189025 25916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:15.189177 25916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:15.189420 25916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:15.189792 25916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:15.189848 25916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.189880 25916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:15.189903 25916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.197296 25916 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:41573
I20260806 03:52:15.197360 26121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:41573 every 8 connection(s)
I20260806 03:52:15.197732 25916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:52:15.201857 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 25916
I20260806 03:52:15.202199 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:43157
--local_ip_for_outbound_sockets=127.12.15.66
--tserver_master_addrs=127.12.15.126:42617
--webserver_port=43933
--webserver_interface=127.12.15.66
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.15.66
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:15.206909 26124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.198456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:35209)
0806 03:52:15.198636 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.198686 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.199199 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.199644 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.199648 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.200223 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.200410 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.200415 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.201542 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.201544 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.202704 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.202708 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.203504 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.203505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.203648 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.204190 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.204197 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.206719 (+ 2522us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
W20260806 03:52:15.207051 26125 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.198512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:35209 (local address 127.12.15.126:42617)
0806 03:52:15.198714 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.198722 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.198784 (+ 62us) server_negotiation.cc:407] Connection header received
0806 03:52:15.199416 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.199421 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.199467 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.199586 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.200538 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.201439 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.202803 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.203400 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.203513 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.206870 (+ 3357us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:35209: BlockingRecv error: recv got EOF from 127.12.15.65:35209 (error 108)
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":82,"threads_started":1}
W20260806 03:52:15.207113 26122 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:42617 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.15.126:42617: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:15.212582 26124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.207319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:59707)
0806 03:52:15.207431 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.207440 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.207473 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.207808 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.207809 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.207949 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.207998 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.208000 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.208837 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.208838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.209613 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.209614 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.210048 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.210048 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.210108 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.210551 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.210556 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.212524 (+ 1968us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98}
W20260806 03:52:15.212688 26125 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.207400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:59707 (local address 127.12.15.126:42617)
0806 03:52:15.207683 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.207685 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.207694 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:15.207716 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.207717 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.207730 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.207795 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.208072 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.208761 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.209699 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.209975 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.210047 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.212634 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:59707: BlockingRecv error: recv got EOF from 127.12.15.65:59707 (error 108)
Metrics: {"server-negotiator.queue_time_us":260}
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:15.217391 26124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.212827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:33831)
0806 03:52:15.212898 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.212908 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.212937 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.213167 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.213168 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.213296 (+ 128us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.213325 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.213327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.214182 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.214183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.214931 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.214934 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.215381 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.215381 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.215422 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.215799 (+ 377us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.215802 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.217350 (+ 1548us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260806 03:52:15.217495 26125 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.212904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:33831 (local address 127.12.15.126:42617)
0806 03:52:15.213053 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.213055 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.213064 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:15.213084 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.213084 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.213093 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.213166 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.213404 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.214069 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.215012 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.215313 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.215372 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.217445 (+ 2073us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:33831: BlockingRecv error: recv got EOF from 127.12.15.65:33831 (error 108)
Metrics: {"server-negotiator.queue_time_us":131}
W20260806 03:52:15.217491 26122 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:52:15.286453 26126 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:15.286710 26126 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:15.286741 26126 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:15.287930 26126 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:15.287972 26126 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:15.287998 26126 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:15.289381 26126 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.66.keytab
--principal=oryx/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:43157
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=43933
--tserver_master_addrs=127.12.15.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26126
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:15.289567 26126 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:15.289763 26126 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:15.294647 26135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:15.294615 26137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:15.294600 26139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:15.294941 26126 server_base.cc:1034] running on GCE node
I20260806 03:52:15.295120 26126 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:15.295358 26126 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:15.296499 26126 hybrid_clock.cc:648] HybridClock initialized: now 1785988335296483 us; error 30 us; skew 500 ppm
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988335, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:15.298972 26126 init.cc:377] Logged in from keytab as oryx/127.12.15.66@KRBTEST.COM (short username oryx)
I20260806 03:52:15.299394 26126 webserver.cc:460] Webserver started at http://127.12.15.66:43933/ using document root <none> and password file <none>
I20260806 03:52:15.299592 26126 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:15.299641 26126 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:15.300869 26126 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:15.301378 26153 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.301529 26126 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.301612 26126 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "f0b91a8ecb6a46698589241aef39595f"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "138042fbfbb206a66e9126f7cbeed85e"
server_key_iv: "28fa525eedd0b008d2e01f09d4264228"
server_key_version: "encryptionkey@0"
I20260806 03:52:15.301940 26126 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:15.318770 26126 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:15.321110 26126 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:15.321255 26126 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:15.321476 26126 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:15.321779 26126 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:15.321812 26126 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.321842 26126 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:15.321859 26126 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.327473 26126 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:43157
I20260806 03:52:15.327523 26327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:43157 every 8 connection(s)
I20260806 03:52:15.327852 26126 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:52:15.328018 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 26126
I20260806 03:52:15.328322 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:41555
--local_ip_for_outbound_sockets=127.12.15.67
--tserver_master_addrs=127.12.15.126:42617
--webserver_port=45193
--webserver_interface=127.12.15.67
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.12.15.67
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:52:15.335119 26125 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.328595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:38431 (local address 127.12.15.126:42617)
0806 03:52:15.328660 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.328662 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.328848 (+ 186us) server_negotiation.cc:407] Connection header received
0806 03:52:15.329323 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.329325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.329341 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.329409 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.330377 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.330872 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.331862 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.332055 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.332111 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.335062 (+ 2951us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:38431: BlockingRecv error: recv got EOF from 127.12.15.66:38431 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:52:15.335173 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.328555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:38431)
0806 03:52:15.328721 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.328762 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.329235 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.329478 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.329482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.330030 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.330282 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.330287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.330962 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.330963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.331772 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.331775 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.332129 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.332130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.332266 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.332738 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.332746 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.334961 (+ 2215us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":93,"threads_started":1}
W20260806 03:52:15.335403 26330 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:42617 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.15.126:42617: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:15.339711 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.335596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:49023)
0806 03:52:15.335637 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.335646 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.335679 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.335836 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.335837 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.335956 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.335999 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.336000 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.336578 (+ 578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.336579 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.337069 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.337070 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.337395 (+ 325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.337396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.337459 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.337876 (+ 417us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.337881 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.339652 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 03:52:15.339800 26125 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.335678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:49023 (local address 127.12.15.126:42617)
0806 03:52:15.335740 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.335742 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.335749 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:15.335769 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.335770 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.335779 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.335823 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.336064 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.336511 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.337142 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.337335 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.337390 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.339744 (+ 2354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:49023: BlockingRecv error: recv got EOF from 127.12.15.66:49023 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:15.343828 26332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.340038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:46303)
0806 03:52:15.340073 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.340081 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.340112 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.340214 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.340215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.340331 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.340353 (+ 22us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.340355 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.340927 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.340928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.341408 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.341409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.341757 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.341758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.341809 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.342182 (+ 373us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.342187 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.343770 (+ 1583us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 03:52:15.343904 26125 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.340099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:46303 (local address 127.12.15.126:42617)
0806 03:52:15.340134 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.340136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.340142 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:52:15.340161 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.340161 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.340169 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.340213 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.340418 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.340857 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.341505 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.341695 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.341747 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.343853 (+ 2106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:46303: BlockingRecv error: recv got EOF from 127.12.15.66:46303 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 03:52:15.344044 26330 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:52:15.411533 26333 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:15.411733 26333 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:15.411755 26333 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:15.412905 26333 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:15.412950 26333 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:15.412979 26333 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:15.414462 26333 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/krb5kdc/oryx_127.12.15.67.keytab
--principal=oryx/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:41555
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=45193
--tserver_master_addrs=127.12.15.126:42617
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26333
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:15.414693 26333 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:15.414917 26333 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:15.419641 26343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:15.419778 26333 server_base.cc:1034] running on GCE node
W20260806 03:52:15.419667 26340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:15.419656 26341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:15.420086 26333 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:15.420315 26333 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:15.421450 26333 hybrid_clock.cc:648] HybridClock initialized: now 1785988335421433 us; error 26 us; skew 500 ppm
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988335, etypes {rep=17 tkt=17 ses=17}, oryx/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:15.423869 26333 init.cc:377] Logged in from keytab as oryx/127.12.15.67@KRBTEST.COM (short username oryx)
I20260806 03:52:15.424278 26333 webserver.cc:460] Webserver started at http://127.12.15.67:45193/ using document root <none> and password file <none>
I20260806 03:52:15.424492 26333 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:15.424557 26333 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:15.425866 26333 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:15.426455 26358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.426606 26333 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.426690 26333 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "c6c8e1a72ed34f58adda940d1e91e1e4"
format_stamp: "Formatted at 2026-08-06 03:52:14 on dist-test-slave-bqcl"
server_key: "9b73c0cdcec19d6ea7c5568748474b24"
server_key_iv: "ebafa87a21ebc86bbf28fd63cc3e23f9"
server_key_version: "encryptionkey@0"
I20260806 03:52:15.427006 26333 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:15.444569 26333 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:15.446610 26333 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:15.446744 26333 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:15.446959 26333 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:15.447243 26333 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:15.447278 26333 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.447305 26333 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:15.447324 26333 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:15.452932 26333 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:41555
I20260806 03:52:15.452992 26524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:41555 every 8 connection(s)
I20260806 03:52:15.453311 26333 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:52:15.453840 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 26333
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
W20260806 03:52:15.461421 26125 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.453985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:44939 (local address 127.12.15.126:42617)
0806 03:52:15.454047 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.454049 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.454318 (+ 269us) server_negotiation.cc:407] Connection header received
0806 03:52:15.454928 (+ 610us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.454929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.454939 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.455001 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.456150 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.456873 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.457999 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.458303 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.458374 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.461358 (+ 2984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:44939: BlockingRecv error: recv got EOF from 127.12.15.67:44939 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:52:15.461632 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.453963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:44939)
0806 03:52:15.454174 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.454226 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.454831 (+ 605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.455064 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.455068 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.455792 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.456053 (+ 261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.456059 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.456952 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.456953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.457914 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.457917 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.458376 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.458377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.458476 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.458909 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.458916 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.461271 (+ 2355us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":84,"threads_started":1}
I20260806 03:52:15.461674 26529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.454386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46032 (local address 127.12.15.126:42617)
0806 03:52:15.454538 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.454539 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.454550 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:52:15.454578 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.454579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.454592 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.454669 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.455307 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.456109 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.457117 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.457418 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.457484 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.458765 (+ 1281us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:15.458769 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:15.458770 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:15.458782 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:15.460244 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:15.460607 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:15.460610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:15.460610 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:15.460753 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:15.460941 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:15.460942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:15.460942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:15.461238 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:15.461368 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:15.461478 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 03:52:15.461590 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":59,"threads_started":1}
W20260806 03:52:15.461874 26525 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:42617 (0 consecutive failures): Not authorized: Failed to ping master at 127.12.15.126:42617: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:15.465943 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.462272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:42687)
0806 03:52:15.462309 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.462319 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.462354 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.462516 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.462517 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.462678 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.462720 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.462722 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.463295 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.463295 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.463785 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.463786 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.464097 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.464098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.464163 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.464474 (+ 311us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.464477 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.465884 (+ 1407us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 03:52:15.466022 26529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.462386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:42687 (local address 127.12.15.126:42617)
0806 03:52:15.462436 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.462438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.462444 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:52:15.462459 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.462460 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.462469 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.462512 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.462791 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.463227 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.463855 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.464039 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.464090 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.465973 (+ 1883us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:42687: BlockingRecv error: recv got EOF from 127.12.15.67:42687 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:15 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:15.469681 26527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:15.466243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:49617)
0806 03:52:15.466331 (+ 88us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:15.466342 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:15.466387 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:15.466489 (+ 102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:15.466489 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:15.466601 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:15.466624 (+ 23us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.466625 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.467196 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.467196 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.467661 (+ 465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:15.467662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.467967 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.467968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:15.468017 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.468312 (+ 295us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:15.468315 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:15.469640 (+ 1325us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":74}
W20260806 03:52:15.469770 26529 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:15.466292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:49617 (local address 127.12.15.126:42617)
0806 03:52:15.466334 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:15.466336 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:15.466374 (+ 38us) server_negotiation.cc:407] Connection header received
0806 03:52:15.466437 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:15.466438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:15.466446 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:15.466485 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:15.466686 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.467118 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.467735 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:15.467909 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:15.467958 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:15.469720 (+ 1762us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:49617: BlockingRecv error: recv got EOF from 127.12.15.67:49617 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260806 03:52:15.469875 26525 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 06 03:52:16 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:16 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:16.222450 26531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:16.217852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:57515)
0806 03:52:16.217988 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:16.217999 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:16.218052 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:16.218199 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:16.218200 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:16.218357 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:16.218407 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:16.218409 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.219182 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.219183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:16.219745 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:16.219746 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.220060 (+ 314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.220060 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:16.220098 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:16.220541 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:16.220546 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:16.222358 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
W20260806 03:52:16.222517 26534 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:16.217932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:57515 (local address 127.12.15.126:42617)
0806 03:52:16.218051 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:16.218053 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:16.218082 (+ 29us) server_negotiation.cc:407] Connection header received
0806 03:52:16.218104 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:16.218104 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:16.218116 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:16.218177 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:16.218470 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.219117 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.219816 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.220008 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.220055 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:16.222455 (+ 2400us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:57515: BlockingRecv error: recv got EOF from 127.12.15.65:57515 (error 108)
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":55,"threads_started":1}
Aug 06 03:52:16 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:16 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:16.348867 26536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:16.344397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:60869)
0806 03:52:16.344525 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:16.344535 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:16.344592 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:16.344708 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:16.344709 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:16.344867 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:16.344916 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:16.344917 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.345620 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.345620 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:16.346264 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:16.346267 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.346683 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.346683 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:16.346739 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:16.347137 (+ 398us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:16.347141 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:16.348799 (+ 1658us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
W20260806 03:52:16.348959 26534 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:16.344482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:60869 (local address 127.12.15.126:42617)
0806 03:52:16.344538 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:16.344540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:16.344577 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:52:16.344632 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:16.344633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:16.344643 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:16.344688 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:16.345036 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.345554 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.346324 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.346612 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.346675 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:16.348887 (+ 2212us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:60869: BlockingRecv error: recv got EOF from 127.12.15.66:60869 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 03:52:16 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:16 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:16.474728 26538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:16.470254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:40199)
0806 03:52:16.470383 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:16.470393 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:16.470443 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:16.470588 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:16.470588 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:16.470750 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:16.470799 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:16.470801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.471462 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.471463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:16.471998 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:16.471999 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.472285 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.472285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:16.472323 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:16.472762 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:16.472766 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:16.474619 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":68,"threads_started":1}
W20260806 03:52:16.474777 26534 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:16.470347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:40199 (local address 127.12.15.126:42617)
0806 03:52:16.470433 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:16.470434 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:16.470446 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:52:16.470507 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:16.470508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:16.470522 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:16.470578 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:16.470890 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.471401 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.472054 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:16.472236 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:16.472278 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:16.474719 (+ 2441us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:40199: BlockingRecv error: recv got EOF from 127.12.15.67:40199 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 06 03:52:17 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:17 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:17.227280 26539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:17.222912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:48401)
0806 03:52:17.223053 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:17.223064 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:17.223111 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:17.223229 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:17.223230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:17.223390 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:17.223439 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:17.223440 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.224035 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.224035 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:17.224571 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:17.224573 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.224910 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.224912 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:17.224974 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:17.225451 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:17.225455 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:17.227202 (+ 1747us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":62,"threads_started":1}
W20260806 03:52:17.227375 26541 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:17.222973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:48401 (local address 127.12.15.126:42617)
0806 03:52:17.223110 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:17.223111 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:17.223119 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:17.223144 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:17.223144 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:17.223154 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:17.223219 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:17.223487 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.223982 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.224628 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.224810 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.224880 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:17.227313 (+ 2433us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:48401: BlockingRecv error: recv got EOF from 127.12.15.65:48401 (error 108)
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":79,"threads_started":1}
Aug 06 03:52:17 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:17 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:17.353456 26542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:17.349309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:51977)
0806 03:52:17.349441 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:17.349451 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:17.349507 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:17.349613 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:17.349614 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:17.349784 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:17.349834 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:17.349835 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.350454 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.350455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:17.350994 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:17.350996 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.351280 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.351281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:17.351319 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:17.351718 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:17.351722 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:17.353391 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":60,"threads_started":1}
W20260806 03:52:17.353556 26541 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:17.349371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:51977 (local address 127.12.15.126:42617)
0806 03:52:17.349423 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:17.349425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:17.349504 (+ 79us) server_negotiation.cc:407] Connection header received
0806 03:52:17.349559 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:17.349559 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:17.349570 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:17.349608 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:17.349882 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.350396 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.351053 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.351233 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.351275 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:17.353490 (+ 2215us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:51977: BlockingRecv error: recv got EOF from 127.12.15.66:51977 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 06 03:52:17 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:17 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:17.479650 26544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:17.475226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:39047)
0806 03:52:17.475399 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:17.475410 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:17.475482 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:17.475653 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:17.475654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:17.475874 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:17.475935 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:17.475937 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.476568 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.476569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:17.477087 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:17.477088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.477367 (+ 279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.477368 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:17.477405 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:17.477829 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:17.477833 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:17.479568 (+ 1735us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":101,"threads_started":1}
W20260806 03:52:17.479732 26541 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:17.475350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:39047 (local address 127.12.15.126:42617)
0806 03:52:17.475414 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:17.475416 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:17.475465 (+ 49us) server_negotiation.cc:407] Connection header received
0806 03:52:17.475548 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:17.475549 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:17.475562 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:17.475636 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:17.476016 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.476508 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.477146 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:17.477318 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:17.477361 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:17.479668 (+ 2307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:39047: BlockingRecv error: recv got EOF from 127.12.15.67:39047 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 06 03:52:18 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:18 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:18.232019 26545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:18.227713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:35673)
0806 03:52:18.227850 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:18.227860 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:18.227908 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:18.228030 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:18.228031 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:18.228194 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:18.228246 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:18.228248 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.228856 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.228856 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:18.229385 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:18.229387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.229719 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.229719 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:18.229763 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:18.230262 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:18.230267 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:18.231961 (+ 1694us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":62,"threads_started":1}
W20260806 03:52:18.232133 26548 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:18.227759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:35673 (local address 127.12.15.126:42617)
0806 03:52:18.227897 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:18.227898 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:18.227907 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:18.227950 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:18.227950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:18.227961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:18.228017 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:18.228297 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.228795 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.229447 (+ 652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.229653 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.229703 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:18.232053 (+ 2350us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:35673: BlockingRecv error: recv got EOF from 127.12.15.65:35673 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":80,"threads_started":1}
Aug 06 03:52:18 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:18 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:18.358364 26549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:18.353957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:43515)
0806 03:52:18.354113 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:18.354148 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:18.354225 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:18.354365 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:18.354366 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:18.354563 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:18.354612 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:18.354614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.355288 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.355289 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:18.355818 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:18.355820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.356155 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.356157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:18.356212 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:18.356616 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:18.356620 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:18.358303 (+ 1683us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
W20260806 03:52:18.358460 26548 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:18.354053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:43515 (local address 127.12.15.126:42617)
0806 03:52:18.354117 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:18.354141 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:18.354213 (+ 72us) server_negotiation.cc:407] Connection header received
0806 03:52:18.354291 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:18.354292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:18.354304 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:18.354348 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:18.354700 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.355210 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.355899 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.356091 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.356147 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:18.358401 (+ 2254us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:43515: BlockingRecv error: recv got EOF from 127.12.15.66:43515 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 06 03:52:18 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:18 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:18.484257 26550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:18.480088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:34503)
0806 03:52:18.480225 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:18.480236 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:18.480294 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:18.480425 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:18.480426 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:18.480592 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:18.480644 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:18.480645 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.481263 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.481264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:18.481789 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:18.481791 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.482060 (+ 269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.482061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:18.482096 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:18.482564 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:18.482568 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:18.484197 (+ 1629us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":69,"threads_started":1}
W20260806 03:52:18.484344 26548 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:18.480177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:34503 (local address 127.12.15.126:42617)
0806 03:52:18.480248 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:18.480250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:18.480278 (+ 28us) server_negotiation.cc:407] Connection header received
0806 03:52:18.480347 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:18.480347 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:18.480363 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:18.480410 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:18.480710 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.481206 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.481844 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:18.482013 (+ 169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:18.482054 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:18.484276 (+ 2222us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:34503: BlockingRecv error: recv got EOF from 127.12.15.67:34503 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 06 03:52:19 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:19 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:19.236939 26551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:19.232460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:56685)
0806 03:52:19.232599 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:19.232613 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:19.232672 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:19.232787 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:19.232788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:19.232951 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:19.233000 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:19.233001 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.233599 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.233599 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:19.234114 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:19.234115 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.234447 (+ 332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.234448 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:19.234486 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:19.234958 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:19.234963 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:19.236863 (+ 1900us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
W20260806 03:52:19.237031 26552 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:19.232528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:56685 (local address 127.12.15.126:42617)
0806 03:52:19.232671 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:19.232673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:19.232682 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:19.232708 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:19.232709 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:19.232719 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:19.232775 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:19.233053 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.233548 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.234199 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.234393 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.234443 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:19.236969 (+ 2526us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:56685: BlockingRecv error: recv got EOF from 127.12.15.65:56685 (error 108)
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":67,"threads_started":1}
Aug 06 03:52:19 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:19 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:19.365737 26553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:19.358844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:57187)
0806 03:52:19.358979 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:19.358991 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:19.359063 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:19.359211 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:19.359212 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:19.359387 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:19.359437 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:19.359439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.360097 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.360098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:19.360628 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:19.360630 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.360973 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.360974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:19.361028 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:19.363904 (+ 2876us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:19.363908 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:19.365674 (+ 1766us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":74,"threads_started":1}
W20260806 03:52:19.365835 26552 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:19.358915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:57187 (local address 127.12.15.126:42617)
0806 03:52:19.358982 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:19.358984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:19.359051 (+ 67us) server_negotiation.cc:407] Connection header received
0806 03:52:19.359122 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:19.359123 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:19.359135 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:19.359182 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:19.359512 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.360015 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.360716 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.360907 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.360963 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:19.365774 (+ 4811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:57187: BlockingRecv error: recv got EOF from 127.12.15.66:57187 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 03:52:19 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:19 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:19.489236 26554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:19.484709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:57149)
0806 03:52:19.484865 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:19.484875 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:19.484943 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:19.485067 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:19.485068 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:19.485316 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:19.485363 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:19.485365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.486012 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.486013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:19.486647 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:19.486650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.486981 (+ 331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.486982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:19.487036 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:19.487515 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:19.487521 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:19.489173 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":94,"threads_started":1}
W20260806 03:52:19.489334 26552 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:19.484769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:57149 (local address 127.12.15.126:42617)
0806 03:52:19.484831 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:19.484833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:19.484934 (+ 101us) server_negotiation.cc:407] Connection header received
0806 03:52:19.485000 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:19.485001 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:19.485014 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:19.485062 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:19.485440 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.485938 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.486725 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:19.486917 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:19.486974 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:19.489271 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:57149: BlockingRecv error: recv got EOF from 127.12.15.67:57149 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 03:52:20 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:20 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:20.241602 26556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:20.237370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:34889)
0806 03:52:20.237487 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:20.237498 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:20.237547 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:20.237664 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:20.237664 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:20.237829 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:20.237878 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:20.237880 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.238502 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.238503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:20.239029 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:20.239030 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.239320 (+ 290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.239321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:20.239358 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:20.239871 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:20.239875 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:20.241541 (+ 1666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":68,"threads_started":1}
W20260806 03:52:20.241706 26557 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:20.237450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:34889 (local address 127.12.15.126:42617)
0806 03:52:20.237556 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:20.237558 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:20.237566 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:20.237585 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:20.237586 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:20.237596 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:20.237653 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:20.237935 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.238452 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.239085 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.239271 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.239314 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:20.241630 (+ 2316us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:34889: BlockingRecv error: recv got EOF from 127.12.15.65:34889 (error 108)
Metrics: {"server-negotiator.queue_time_us":85,"thread_start_us":56,"threads_started":1}
Aug 06 03:52:20 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:20 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:20.370338 26558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:20.366199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:36545)
0806 03:52:20.366312 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:20.366322 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:20.366377 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:20.366491 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:20.366492 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:20.366649 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:20.366698 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:20.366700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.367295 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.367296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:20.367810 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:20.367811 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.368087 (+ 276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.368088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:20.368122 (+ 34us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:20.368594 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:20.368598 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:20.370278 (+ 1680us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":61,"threads_started":1}
W20260806 03:52:20.370426 26557 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:20.366300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:36545 (local address 127.12.15.126:42617)
0806 03:52:20.366346 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:20.366348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:20.366360 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:52:20.366433 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:20.366433 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:20.366445 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:20.366478 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:20.366749 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.367243 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.367867 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.368041 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.368082 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:20.370368 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:36545: BlockingRecv error: recv got EOF from 127.12.15.66:36545 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
Aug 06 03:52:20 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:20 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:20.494213 26559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:20.489686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:60923)
0806 03:52:20.489813 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:20.489824 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:20.489885 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:20.490052 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:20.490053 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:20.490291 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:20.490352 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:20.490355 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.491028 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.491028 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:20.491633 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:20.491635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.491971 (+ 336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.491971 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:20.492008 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:20.492465 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:20.492469 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:20.494144 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
W20260806 03:52:20.494315 26557 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:20.489774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:60923 (local address 127.12.15.126:42617)
0806 03:52:20.489854 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:20.489856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:20.489870 (+ 14us) server_negotiation.cc:407] Connection header received
0806 03:52:20.489954 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:20.489954 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:20.489969 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:20.490033 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:20.490434 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.490953 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.491719 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:20.491912 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:20.491968 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:20.494254 (+ 2286us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:60923: BlockingRecv error: recv got EOF from 127.12.15.67:60923 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 06 03:52:21 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:21 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:21.246556 26560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:21.242054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:45469)
0806 03:52:21.242213 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:21.242225 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:21.242277 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:21.242400 (+ 123us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:21.242401 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:21.242561 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:21.242610 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:21.242612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.243220 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.243221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:21.243746 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:21.243747 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.244067 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.244068 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:21.244104 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:21.244528 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:21.244532 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:21.246496 (+ 1964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":88,"threads_started":1}
W20260806 03:52:21.246673 26563 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:21.242142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:45469 (local address 127.12.15.126:42617)
0806 03:52:21.242275 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:21.242277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:21.242287 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:21.242328 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:21.242328 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:21.242340 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:21.242389 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:21.242675 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.243166 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.243816 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.244012 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.244065 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:21.246597 (+ 2532us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:45469: BlockingRecv error: recv got EOF from 127.12.15.65:45469 (error 108)
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":74,"threads_started":1}
Aug 06 03:52:21 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:21 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:21.375206 26564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:21.370799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:35423)
0806 03:52:21.370940 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:21.370951 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:21.371020 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:21.371154 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:21.371155 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:21.371332 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:21.371380 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:21.371382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.372022 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.372023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:21.372625 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:21.372628 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.372950 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.372952 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:21.373008 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:21.373419 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:21.373423 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:21.375127 (+ 1704us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
W20260806 03:52:21.375294 26563 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:21.370866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:35423 (local address 127.12.15.126:42617)
0806 03:52:21.370932 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:21.370934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:21.371016 (+ 82us) server_negotiation.cc:407] Connection header received
0806 03:52:21.371083 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:21.371084 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:21.371097 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:21.371144 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:21.371452 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.371945 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.372695 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.372890 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.372944 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:21.375241 (+ 2297us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:35423: BlockingRecv error: recv got EOF from 127.12.15.66:35423 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 06 03:52:21 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:21 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:21.498839 26566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:21.494652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:33563)
0806 03:52:21.494823 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:21.494835 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:21.494899 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:21.495014 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:21.495015 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:21.495183 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:21.495232 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:21.495234 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.495819 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.495819 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:21.496343 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:21.496344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.496615 (+ 271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.496615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:21.496665 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:21.497105 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:21.497109 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:21.498778 (+ 1669us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":106,"threads_started":1}
W20260806 03:52:21.498921 26563 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:21.494723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:33563 (local address 127.12.15.126:42617)
0806 03:52:21.494785 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:21.494787 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:21.494893 (+ 106us) server_negotiation.cc:407] Connection header received
0806 03:52:21.494951 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:21.494951 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:21.494963 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:21.495000 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:21.495283 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.495764 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.496399 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:21.496570 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:21.496610 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:21.498865 (+ 2255us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:33563: BlockingRecv error: recv got EOF from 127.12.15.67:33563 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 03:52:22 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:22 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:22.251786 26567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:22.247086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:57099)
0806 03:52:22.247244 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:22.247256 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:22.247322 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:22.247476 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:22.247477 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:22.247675 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:22.247725 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:22.247728 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.248411 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.248411 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:22.249022 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:22.249025 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.249444 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.249445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:22.249492 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:22.249962 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:22.249967 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:22.251723 (+ 1756us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":78,"threads_started":1}
W20260806 03:52:22.251883 26572 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:22.247172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:57099 (local address 127.12.15.126:42617)
0806 03:52:22.247275 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:22.247277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:22.247305 (+ 28us) server_negotiation.cc:407] Connection header received
0806 03:52:22.247386 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:22.247387 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:22.247398 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:22.247456 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:22.247816 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.248349 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.249088 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.249383 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.249435 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:22.251813 (+ 2378us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:57099: BlockingRecv error: recv got EOF from 127.12.15.65:57099 (error 108)
Metrics: {"server-negotiator.queue_time_us":81,"thread_start_us":51,"threads_started":1}
Aug 06 03:52:22 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:22 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:22.380370 26573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:22.375705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:51615)
0806 03:52:22.375856 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:22.375873 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:22.375935 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:22.376061 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:22.376062 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:22.376227 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:22.376277 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:22.376279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.377026 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.377028 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:22.377557 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:22.377559 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.377856 (+ 297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.377856 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:22.377894 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:22.378394 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:22.378399 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:22.380278 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
W20260806 03:52:22.380478 26572 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:22.375770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:51615 (local address 127.12.15.126:42617)
0806 03:52:22.375825 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:22.375827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:22.375930 (+ 103us) server_negotiation.cc:407] Connection header received
0806 03:52:22.375982 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:22.375983 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:22.375994 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:22.376046 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:22.376328 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.376962 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.377618 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.377804 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.377850 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:22.380409 (+ 2559us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:51615: BlockingRecv error: recv got EOF from 127.12.15.66:51615 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 03:52:22 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:22 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:22.503671 26574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:22.499298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:49673)
0806 03:52:22.499430 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:22.499441 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:22.499500 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:22.499619 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:22.499620 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:22.499783 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:22.499833 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:22.499835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.500449 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.500449 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:22.500985 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:22.500986 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.501343 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.501344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:22.501395 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:22.501815 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:22.501819 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:22.503610 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
W20260806 03:52:22.503782 26572 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:22.499369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:49673 (local address 127.12.15.126:42617)
0806 03:52:22.499424 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:22.499426 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:22.499492 (+ 66us) server_negotiation.cc:407] Connection header received
0806 03:52:22.499545 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:22.499545 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:22.499558 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:22.499606 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:22.499881 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.500377 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.501052 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:22.501247 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:22.501301 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:22.503711 (+ 2410us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:49673: BlockingRecv error: recv got EOF from 127.12.15.67:49673 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
Aug 06 03:52:23 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:23 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:23.256690 26575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:23.252277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:56727)
0806 03:52:23.252446 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:23.252459 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:23.252524 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:23.252653 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:23.252654 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:23.252815 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:23.252866 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:23.252867 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.253478 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.253478 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:23.254004 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:23.254006 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.254449 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.254450 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:23.254513 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:23.254922 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:23.254926 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:23.256611 (+ 1685us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
W20260806 03:52:23.256794 26576 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:23.252408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:56727 (local address 127.12.15.126:42617)
0806 03:52:23.252538 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:23.252541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:23.252549 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:23.252572 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:23.252573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:23.252583 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:23.252639 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:23.252919 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.253425 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.254091 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.254380 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.254445 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:23.256725 (+ 2280us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:56727: BlockingRecv error: recv got EOF from 127.12.15.65:56727 (error 108)
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":70,"threads_started":1}
Aug 06 03:52:23 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:23 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:23.385612 26579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:23.380810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:44811)
0806 03:52:23.380972 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:23.380983 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:23.381037 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:23.381172 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:23.381173 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:23.381349 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:23.381400 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:23.381402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.382185 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.382186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:23.382931 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:23.382936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.383275 (+ 339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.383276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:23.383323 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:23.383727 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:23.383731 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:23.385530 (+ 1799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":71,"threads_started":1}
W20260806 03:52:23.385715 26576 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:23.380871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:44811 (local address 127.12.15.126:42617)
0806 03:52:23.380929 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:23.380931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:23.381033 (+ 102us) server_negotiation.cc:407] Connection header received
0806 03:52:23.381096 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:23.381097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:23.381109 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:23.381149 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:23.381455 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.382073 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.383008 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.383201 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.383247 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:23.385661 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:44811: BlockingRecv error: recv got EOF from 127.12.15.66:44811 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 06 03:52:23 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:23 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:23.508663 26580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:23.504129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:50165)
0806 03:52:23.504280 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:23.504291 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:23.504358 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:23.504490 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:23.504491 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:23.504693 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:23.504742 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:23.504744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.505471 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.505472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:23.506048 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:23.506050 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.506420 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.506421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:23.506480 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:23.506879 (+ 399us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:23.506883 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:23.508602 (+ 1719us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
W20260806 03:52:23.508788 26576 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:23.504213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:50165 (local address 127.12.15.126:42617)
0806 03:52:23.504273 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:23.504275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:23.504348 (+ 73us) server_negotiation.cc:407] Connection header received
0806 03:52:23.504416 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:23.504417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:23.504431 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:23.504475 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:23.504813 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.505391 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.506150 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:23.506355 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:23.506413 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:23.508715 (+ 2302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:50165: BlockingRecv error: recv got EOF from 127.12.15.67:50165 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 03:52:24 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:24 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:24.261447 26581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:24.257124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:32879)
0806 03:52:24.257268 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:24.257278 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:24.257326 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:24.257457 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:24.257458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:24.257620 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:24.257684 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:24.257686 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.258325 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.258326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:24.258856 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:24.258857 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.259162 (+ 305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.259163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:24.259209 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:24.259613 (+ 404us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:24.259617 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:24.261375 (+ 1758us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
W20260806 03:52:24.261539 26583 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:24.257179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:32879 (local address 127.12.15.126:42617)
0806 03:52:24.257314 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:24.257316 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:24.257324 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:52:24.257376 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:24.257376 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:24.257387 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:24.257445 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:24.257736 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.258273 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.258912 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.259098 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.259140 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:24.261472 (+ 2332us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:32879: BlockingRecv error: recv got EOF from 127.12.15.65:32879 (error 108)
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":63,"threads_started":1}
Aug 06 03:52:24 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:24 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:24.395373 26585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:24.386097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:45911)
0806 03:52:24.386271 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:24.386283 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:24.386359 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:24.386563 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:24.386564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:24.386788 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:24.386852 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:24.386854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.387529 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.387530 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:24.388089 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:24.388091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.388454 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.388455 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:24.388514 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:24.393201 (+ 4687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:24.393207 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:24.395275 (+ 2068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":93,"threads_started":1}
W20260806 03:52:24.395407 26583 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:24.386179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:45911 (local address 127.12.15.126:42617)
0806 03:52:24.386238 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:24.386241 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:24.386346 (+ 105us) server_negotiation.cc:407] Connection header received
0806 03:52:24.386426 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:24.386426 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:24.386440 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:24.386543 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:24.386944 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.387444 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.388180 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.388381 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.388447 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:24.395353 (+ 6906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:45911: BlockingRecv error: recv got EOF from 127.12.15.66:45911 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 06 03:52:24 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:24 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:24.513680 26587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:24.509138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:37313)
0806 03:52:24.509300 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:24.509311 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:24.509369 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:24.509487 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:24.509488 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:24.509663 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:24.509714 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:24.509715 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.510347 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.510348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:24.510907 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:24.510910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.511203 (+ 293us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.511203 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:24.511249 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:24.511741 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:24.511747 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:24.513599 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":97,"threads_started":1}
W20260806 03:52:24.513768 26583 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:24.509218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:37313 (local address 127.12.15.126:42617)
0806 03:52:24.509288 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:24.509290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:24.509362 (+ 72us) server_negotiation.cc:407] Connection header received
0806 03:52:24.509415 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:24.509415 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:24.509427 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:24.509477 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:24.509761 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.510273 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.510976 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:24.511150 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:24.511199 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:24.513710 (+ 2511us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:37313: BlockingRecv error: recv got EOF from 127.12.15.67:37313 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 06 03:52:25 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:25 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:25.266714 26588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:25.261954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:34363)
0806 03:52:25.262089 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:25.262100 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:25.262192 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:25.262329 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:25.262330 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:25.262503 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:25.262566 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:25.262569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.263427 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.263428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:25.263979 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:25.263981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.264328 (+ 347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.264329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:25.264369 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:25.264845 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:25.264850 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:25.266654 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":75,"threads_started":1}
W20260806 03:52:25.266840 26589 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:25.262052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:34363 (local address 127.12.15.126:42617)
0806 03:52:25.262193 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:25.262195 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:25.262206 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:52:25.262247 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:25.262247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:25.262258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:25.262316 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:25.262673 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.263354 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.264059 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.264270 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.264323 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:25.266758 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:34363: BlockingRecv error: recv got EOF from 127.12.15.65:34363 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":50,"threads_started":1}
Aug 06 03:52:25 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:25 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:25.400039 26594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:25.395849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:47959)
0806 03:52:25.395983 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:25.395994 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:25.396052 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:25.396178 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:25.396179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:25.396347 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:25.396398 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:25.396400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.397010 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.397011 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:25.397534 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:25.397536 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.397854 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.397855 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:25.397892 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:25.398331 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:25.398335 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:25.399976 (+ 1641us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":75,"threads_started":1}
W20260806 03:52:25.400131 26589 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:25.395908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:47959 (local address 127.12.15.126:42617)
0806 03:52:25.395962 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:25.395964 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:25.396048 (+ 84us) server_negotiation.cc:407] Connection header received
0806 03:52:25.396108 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:25.396109 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:25.396120 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:25.396158 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:25.396448 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.396950 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.397605 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.397802 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.397850 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:25.400071 (+ 2221us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:47959: BlockingRecv error: recv got EOF from 127.12.15.66:47959 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 03:52:25 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:25 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:25.518571 26595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:25.514205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:34723)
0806 03:52:25.514344 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:25.514354 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:25.514404 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:25.514511 (+ 107us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:25.514511 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:25.514683 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:25.514744 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:25.514746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.515395 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.515395 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:25.515929 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:25.515930 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.516214 (+ 284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.516215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:25.516253 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:25.516718 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:25.516723 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:25.518509 (+ 1786us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":63,"threads_started":1}
W20260806 03:52:25.518693 26589 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:25.514337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:34723 (local address 127.12.15.126:42617)
0806 03:52:25.514399 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:25.514401 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:25.514411 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:25.514438 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:25.514439 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:25.514450 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:25.514500 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:25.514824 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.515338 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.515984 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:25.516165 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:25.516207 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:25.518603 (+ 2396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:34723: BlockingRecv error: recv got EOF from 127.12.15.67:34723 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 03:52:26 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:26 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:26.271706 26597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:26.267172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:60935)
0806 03:52:26.267333 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:26.267345 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:26.267402 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:26.267537 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:26.267538 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:26.267724 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:26.267772 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:26.267773 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.268399 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.268400 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:26.268976 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:26.268978 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.269331 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.269332 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:26.269373 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:26.269845 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:26.269849 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:26.271620 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":72,"threads_started":1}
W20260806 03:52:26.271777 26599 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:26.267260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:60935 (local address 127.12.15.126:42617)
0806 03:52:26.267377 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:26.267379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:26.267388 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:26.267446 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:26.267447 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:26.267457 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:26.267521 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:26.267830 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.268330 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.269063 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.269270 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.269324 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:26.271718 (+ 2394us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:60935: BlockingRecv error: recv got EOF from 127.12.15.65:60935 (error 108)
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":57,"threads_started":1}
Aug 06 03:52:26 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:26 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:26.405153 26600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:26.400495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:41875)
0806 03:52:26.400655 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:26.400666 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:26.400727 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:26.400847 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:26.400848 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:26.401025 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:26.401076 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:26.401077 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.401671 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.401671 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:26.402406 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:26.402410 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.402730 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.402730 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:26.402769 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:26.403178 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:26.403182 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:26.405075 (+ 1893us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
W20260806 03:52:26.405259 26599 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:26.400573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:41875 (local address 127.12.15.126:42617)
0806 03:52:26.400665 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:26.400667 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:26.400720 (+ 53us) server_negotiation.cc:407] Connection header received
0806 03:52:26.400776 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:26.400777 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:26.400788 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:26.400837 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:26.401123 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.401608 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.402465 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.402673 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.402727 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:26.405183 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:41875: BlockingRecv error: recv got EOF from 127.12.15.66:41875 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 06 03:52:26 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:26 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:26.525334 26601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:26.519109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:39365)
0806 03:52:26.519299 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:26.519321 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:26.519395 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:26.519586 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:26.519587 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:26.519849 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:26.519915 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:26.519917 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.520713 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.520714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:26.521414 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:26.521417 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.521830 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.521831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:26.521892 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:26.522684 (+ 792us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:26.522689 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:26.525250 (+ 2561us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":106,"threads_started":1}
W20260806 03:52:26.525472 26599 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:26.519192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:39365 (local address 127.12.15.126:42617)
0806 03:52:26.519276 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:26.519278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:26.519413 (+ 135us) server_negotiation.cc:407] Connection header received
0806 03:52:26.519448 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:26.519449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:26.519468 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:26.519566 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:26.519992 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.520633 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.521523 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:26.521752 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:26.521825 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:26.525371 (+ 3546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:39365: BlockingRecv error: recv got EOF from 127.12.15.67:39365 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 06 03:52:27 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:27 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:27.278983 26602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:27.272229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:44653)
0806 03:52:27.272393 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:27.272407 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:27.272474 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:27.272731 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:27.272733 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:27.273037 (+ 304us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:27.273116 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:27.273119 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.273961 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.273962 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:27.274784 (+ 822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:27.274787 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.275268 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.275269 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:27.275332 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:27.276036 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:27.276046 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:27.278901 (+ 2855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
W20260806 03:52:27.279197 26605 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:27.272341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:44653 (local address 127.12.15.126:42617)
0806 03:52:27.272487 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:27.272492 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:27.272501 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:27.272555 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:27.272555 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:27.272571 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:27.272664 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:27.273229 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.273865 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.274890 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.275173 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.275261 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:27.279064 (+ 3803us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:44653: BlockingRecv error: recv got EOF from 127.12.15.65:44653 (error 108)
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":80,"threads_started":1}
Aug 06 03:52:27 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:27 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:27.412060 26607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:27.405780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:51127)
0806 03:52:27.405968 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:27.405990 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:27.406084 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:27.406308 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:27.406309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:27.406597 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:27.406669 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:27.406671 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.407442 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.407443 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:27.408199 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:27.408206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.408634 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.408635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:27.408721 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:27.409342 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:27.409346 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:27.411961 (+ 2615us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":95,"threads_started":1}
W20260806 03:52:27.412147 26605 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:27.405865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:51127 (local address 127.12.15.126:42617)
0806 03:52:27.405961 (+ 96us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:27.405965 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:27.406066 (+ 101us) server_negotiation.cc:407] Connection header received
0806 03:52:27.406193 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:27.406194 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:27.406210 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:27.406274 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:27.406748 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.407355 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.408298 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.408532 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.408611 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:27.412082 (+ 3471us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:51127: BlockingRecv error: recv got EOF from 127.12.15.66:51127 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 06 03:52:27 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:27 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:27.531917 26609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:27.525953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:42407)
0806 03:52:27.526159 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:27.526174 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:27.526252 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:27.526414 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:27.526415 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:27.526649 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:27.526733 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:27.526735 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.527606 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.527606 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:27.528281 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:27.528283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.528691 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.528691 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:27.528745 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:27.529495 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:27.529507 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:27.531836 (+ 2329us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":99,"threads_started":1}
W20260806 03:52:27.532166 26605 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:27.526033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:42407 (local address 127.12.15.126:42617)
0806 03:52:27.526108 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:27.526111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:27.526239 (+ 128us) server_negotiation.cc:407] Connection header received
0806 03:52:27.526311 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:27.526311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:27.526324 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:27.526390 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:27.526847 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.527480 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.528365 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:27.528614 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:27.528681 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:27.531991 (+ 3310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:42407: BlockingRecv error: recv got EOF from 127.12.15.67:42407 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 06 03:52:28 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:28 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:28.283762 26611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:28.279522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:41119)
0806 03:52:28.279696 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:28.279706 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:28.279767 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:28.279882 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:28.279883 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:28.280050 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:28.280101 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:28.280103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.280762 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.280763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:28.281293 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:28.281295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.281628 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.281628 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:28.281670 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:28.282088 (+ 418us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:28.282092 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:28.283703 (+ 1611us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":93,"threads_started":1}
W20260806 03:52:28.283861 26612 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:28.279569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:41119 (local address 127.12.15.126:42617)
0806 03:52:28.279707 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:28.279709 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:28.279749 (+ 40us) server_negotiation.cc:407] Connection header received
0806 03:52:28.279808 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:28.279808 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:28.279820 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:28.279873 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:28.280152 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.280697 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.281371 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.281572 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.281626 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:28.283792 (+ 2166us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:41119: BlockingRecv error: recv got EOF from 127.12.15.65:41119 (error 108)
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":69,"threads_started":1}
Aug 06 03:52:28 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:28 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:28.417064 26613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:28.412608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:47425)
0806 03:52:28.412784 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:28.412797 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:28.412851 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:28.412970 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:28.412971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:28.413141 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:28.413202 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:28.413204 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.413809 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.413809 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:28.414363 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:28.414365 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.414666 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.414667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:28.414702 (+ 35us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:28.415162 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:28.415166 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:28.416974 (+ 1808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":83,"threads_started":1}
W20260806 03:52:28.417133 26612 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:28.412786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:47425 (local address 127.12.15.126:42617)
0806 03:52:28.412841 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:28.412843 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:28.412854 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:52:28.412905 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:28.412906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:28.412917 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:28.412956 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:28.413261 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.413753 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.414420 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.414614 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.414663 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:28.417079 (+ 2416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:47425: BlockingRecv error: recv got EOF from 127.12.15.66:47425 (error 108)
Metrics: {"server-negotiator.queue_time_us":32}
Aug 06 03:52:28 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:28 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:28.536943 26614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:28.532411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:45401)
0806 03:52:28.532574 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:28.532584 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:28.532655 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:28.532793 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:28.532794 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:28.532982 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:28.533031 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:28.533033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.533724 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.533725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:28.534301 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:28.534303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.534623 (+ 320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.534624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:28.534679 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:28.535123 (+ 444us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:28.535128 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:28.536851 (+ 1723us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":73,"threads_started":1}
W20260806 03:52:28.537014 26612 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:28.532476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:45401 (local address 127.12.15.126:42617)
0806 03:52:28.532538 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:28.532540 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:28.532648 (+ 108us) server_negotiation.cc:407] Connection header received
0806 03:52:28.532717 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:28.532718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:28.532731 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:28.532782 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:28.533106 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.533649 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.534377 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:28.534563 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:28.534618 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:28.536959 (+ 2341us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:45401: BlockingRecv error: recv got EOF from 127.12.15.67:45401 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 06 03:52:29 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:29 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:29.288785 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:29.284206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:51591)
0806 03:52:29.284367 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:29.284378 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:29.284442 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:29.284599 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:29.284600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:29.284773 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:29.284821 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:29.284823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.285447 (+ 624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.285447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:29.286061 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:29.286063 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.286441 (+ 378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.286442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:29.286481 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:29.286997 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:29.287001 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:29.288714 (+ 1713us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":100,"threads_started":1}
W20260806 03:52:29.288874 26616 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:29.284286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:51591 (local address 127.12.15.126:42617)
0806 03:52:29.284394 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:29.284396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:29.284439 (+ 43us) server_negotiation.cc:407] Connection header received
0806 03:52:29.284483 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:29.284484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:29.284495 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:29.284549 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:29.284870 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.285389 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.286141 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.286383 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.286428 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:29.288801 (+ 2373us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:51591: BlockingRecv error: recv got EOF from 127.12.15.65:51591 (error 108)
Metrics: {"server-negotiator.queue_time_us":88,"thread_start_us":49,"threads_started":1}
Aug 06 03:52:29 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:29 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:29.425693 26617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:29.417532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:48541)
0806 03:52:29.417697 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:29.417709 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:29.417780 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:29.417908 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:29.417908 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:29.418097 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:29.418183 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:29.418184 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.418793 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.418794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:29.419341 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:29.419343 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.419626 (+ 283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.419627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:29.419665 (+ 38us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:29.423859 (+ 4194us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:29.423864 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:29.425628 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
W20260806 03:52:29.425792 26616 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:29.417651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:48541 (local address 127.12.15.126:42617)
0806 03:52:29.417713 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:29.417714 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:29.417769 (+ 55us) server_negotiation.cc:407] Connection header received
0806 03:52:29.417831 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:29.417831 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:29.417842 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:29.417893 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:29.418234 (+ 341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.418735 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.419404 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.419580 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.419619 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:29.425730 (+ 6111us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:48541: BlockingRecv error: recv got EOF from 127.12.15.66:48541 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 06 03:52:29 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:29 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.67@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:29.542035 26619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:29.537444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.67:39755)
0806 03:52:29.537609 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:29.537621 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:29.537680 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:29.537821 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:29.537822 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:29.538023 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:29.538067 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:29.538069 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.538746 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.538747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:29.539307 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:29.539309 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.539639 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.539640 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:29.539695 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:29.540141 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:29.540146 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:29.541970 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":74,"threads_started":1}
W20260806 03:52:29.542160 26616 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:29.537509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:39755 (local address 127.12.15.126:42617)
0806 03:52:29.537557 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:29.537559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:29.537682 (+ 123us) server_negotiation.cc:407] Connection header received
0806 03:52:29.537728 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:29.537728 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:29.537743 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:29.537799 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:29.538157 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.538673 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.539385 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:29.539581 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:29.539636 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:29.542073 (+ 2437us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.67:39755: BlockingRecv error: recv got EOF from 127.12.15.67:39755 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
Aug 06 03:52:30 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:30 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.65@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:30.293512 26620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:30.289237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.65:42697)
0806 03:52:30.289374 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:30.289382 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:30.289422 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:30.289604 (+ 182us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:30.289605 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:30.289773 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:30.289821 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:30.289823 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.290480 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.290480 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:30.291013 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:30.291015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.291334 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.291335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:30.291372 (+ 37us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.291772 (+ 400us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:30.291776 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:30.293451 (+ 1675us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":61,"threads_started":1}
W20260806 03:52:30.293632 26621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:30.289343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:42697 (local address 127.12.15.126:42617)
0806 03:52:30.289464 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:30.289466 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:30.289473 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:52:30.289493 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:30.289493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:30.289521 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:30.289593 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:30.289879 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.290409 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.291082 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.291282 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.291332 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.293550 (+ 2218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.65:42697: BlockingRecv error: recv got EOF from 127.12.15.65:42697 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
Aug 06 03:52:30 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
Aug 06 03:52:30 dist-test-slave-bqcl krb5kdc[25048](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.12.15.66@KRBTEST.COM for oryx/127.12.15.126@KRBTEST.COM, Server not found in Kerberos database
I20260806 03:52:30.430729 26624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:30.426228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42617 (local address 127.12.15.66:44871)
0806 03:52:30.426357 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:30.426368 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:30.426434 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:30.426563 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:30.426564 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:30.426739 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:30.426796 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:30.426798 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.427448 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.427449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:30.427992 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:30.427994 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.428324 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.428325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:30.428381 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.428819 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:30.428823 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:30.430640 (+ 1817us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.12.15.126:42617: Server oryx/127.12.15.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":68,"threads_started":1}
W20260806 03:52:30.430807 26621 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:30.426319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:44871 (local address 127.12.15.126:42617)
0806 03:52:30.426371 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:30.426373 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:30.426414 (+ 41us) server_negotiation.cc:407] Connection header received
0806 03:52:30.426493 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:30.426493 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:30.426506 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:30.426551 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:30.426865 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.427369 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.428068 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.428263 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.428318 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.430753 (+ 2435us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.12.15.66:44871: BlockingRecv error: recv got EOF from 127.12.15.66:44871 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:52:30.454133 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 25916
I20260806 03:52:30.471212 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 26126
I20260806 03:52:30.480350 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 26333
I20260806 03:52:30.486748 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 25808
2026-08-06T03:52:30Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17536 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-06T03:52:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:30Z Disabled control of system clock
I20260806 03:52:30.517961 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:46651
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:46651
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260806 03:52:30.597483 26629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:30.597697 26629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:30.597723 26629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:30.598879 26629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:30.598919 26629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:30.598937 26629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:30.598953 26629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:30.598968 26629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:30.600271 26629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:46651
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:46651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26629
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:30.600508 26629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:30.600775 26629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:30.605583 26640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:30.605654 26629 server_base.cc:1034] running on GCE node
W20260806 03:52:30.605607 26637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:30.605564 26638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:30.606076 26629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:30.606370 26629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:30.607504 26629 hybrid_clock.cc:648] HybridClock initialized: now 1785988350607489 us; error 39 us; skew 500 ppm
I20260806 03:52:30.608671 26629 webserver.cc:460] Webserver started at http://127.12.15.126:41475/ using document root <none> and password file <none>
I20260806 03:52:30.608875 26629 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:30.608923 26629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:30.609076 26629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:30.610113 26629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "f87251b76c754267ba7b1301c273b739"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "0c1195b99ce4bc5c917c3cda60217d17"
server_key_iv: "ac744612062d9ab27f8ff0e2344f0f59"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.610499 26629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "f87251b76c754267ba7b1301c273b739"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "0c1195b99ce4bc5c917c3cda60217d17"
server_key_iv: "ac744612062d9ab27f8ff0e2344f0f59"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.611737 26629 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:52:30.612407 26648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:30.612576 26629 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:30.612658 26629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "f87251b76c754267ba7b1301c273b739"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "0c1195b99ce4bc5c917c3cda60217d17"
server_key_iv: "ac744612062d9ab27f8ff0e2344f0f59"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.612725 26629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:30.627846 26629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:30.628127 26629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:30.628253 26629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:30.632086 26629 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:46651
I20260806 03:52:30.632124 26725 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:46651 every 8 connection(s)
I20260806 03:52:30.632421 26629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:52:30.633061 26726 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:30.635603 26726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739: Bootstrap starting.
I20260806 03:52:30.636173 26726 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:30.636439 26726 log.cc:824] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:30.637045 26726 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739: No bootstrap required, opened a new log
I20260806 03:52:30.638600 26726 raft_consensus.cc:348] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87251b76c754267ba7b1301c273b739" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46651 } }
I20260806 03:52:30.638746 26726 raft_consensus.cc:374] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:30.638780 26726 raft_consensus.cc:729] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f87251b76c754267ba7b1301c273b739, State: Initialized, Role: FOLLOWER
I20260806 03:52:30.638917 26726 consensus_queue.cc:260] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87251b76c754267ba7b1301c273b739" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46651 } }
I20260806 03:52:30.638974 26726 raft_consensus.cc:388] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:52:30.638996 26726 raft_consensus.cc:482] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:52:30.639040 26726 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:30.639518 26726 raft_consensus.cc:504] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87251b76c754267ba7b1301c273b739" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46651 } }
I20260806 03:52:30.639603 26726 leader_election.cc:302] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f87251b76c754267ba7b1301c273b739; no voters:
I20260806 03:52:30.639750 26726 leader_election.cc:288] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:52:30.639813 26730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:30.639964 26726 sys_catalog.cc:564] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:30.640027 26730 raft_consensus.cc:686] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [term 1 LEADER]: Becoming Leader. State: Replica: f87251b76c754267ba7b1301c273b739, State: Running, Role: LEADER
I20260806 03:52:30.640172 26730 consensus_queue.cc:237] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87251b76c754267ba7b1301c273b739" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46651 } }
I20260806 03:52:30.640671 26732 sys_catalog.cc:455] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f87251b76c754267ba7b1301c273b739. Latest consensus state: current_term: 1 leader_uuid: "f87251b76c754267ba7b1301c273b739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87251b76c754267ba7b1301c273b739" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46651 } } }
I20260806 03:52:30.640774 26732 sys_catalog.cc:458] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:30.640671 26731 sys_catalog.cc:455] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f87251b76c754267ba7b1301c273b739" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f87251b76c754267ba7b1301c273b739" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46651 } } }
I20260806 03:52:30.640838 26731 sys_catalog.cc:458] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739 [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:30.640996 26744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:30.641521 26744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:30.642231 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 26629
I20260806 03:52:30.642329 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:52:30.642535 12349 external_mini_cluster.cc:1428] Setting key 263bbf93b6ce9676bb5616f04a0b573d
I20260806 03:52:30.643069 26744 catalog_manager.cc:1347] Generated new cluster ID: 6d9b071d742d4e3988b71e92e53eb11a
I20260806 03:52:30.643121 26744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:52:30.648445 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:30.643018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33142 (local address 127.12.15.126:46651)
0806 03:52:30.643208 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:30.643217 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:30.643256 (+ 39us) server_negotiation.cc:407] Connection header received
0806 03:52:30.643366 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:30.643373 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:30.643437 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:30.643569 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:30.643783 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.645139 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.646286 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.646936 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.647062 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.647729 (+ 667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:30.647733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:30.647733 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:52:30.647938 (+ 205us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:30.647984 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:52:30.647999 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:30.648069 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:30.648134 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 03:52:30.648231 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260806 03:52:30.657631 26744 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:52:30.658087 26744 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:52:30.671190 26744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f87251b76c754267ba7b1301c273b739: Generated new TSK 0
I20260806 03:52:30.671376 26744 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:52:30.675297 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:46651
--builtin_ntp_servers=127.12.15.84:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 03:52:30.759300 26764 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:30.759497 26764 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:30.759524 26764 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:30.760685 26764 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:30.760725 26764 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:30.760747 26764 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:30.762058 26764 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:46651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26764
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:30.762305 26764 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:30.762598 26764 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:30.767364 26771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:30.767530 26764 server_base.cc:1034] running on GCE node
W20260806 03:52:30.767386 26774 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:30.767416 26772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:30.767874 26764 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:30.768103 26764 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:30.769259 26764 hybrid_clock.cc:648] HybridClock initialized: now 1785988350769243 us; error 29 us; skew 500 ppm
I20260806 03:52:30.770555 26764 webserver.cc:460] Webserver started at http://127.12.15.65:46201/ using document root <none> and password file <none>
I20260806 03:52:30.770848 26764 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:30.770912 26764 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:30.771063 26764 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:30.772125 26764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "08cbae34ed154c9684cbc8a024e130fe"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "f66df795ce291d95bc8fd48010cecf71"
server_key_iv: "04b7478ea671223305fefcbb1310a69a"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.772449 26764 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "08cbae34ed154c9684cbc8a024e130fe"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "f66df795ce291d95bc8fd48010cecf71"
server_key_iv: "04b7478ea671223305fefcbb1310a69a"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.773722 26764 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:30.774420 26792 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:30.774559 26764 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:30.774636 26764 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "08cbae34ed154c9684cbc8a024e130fe"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "f66df795ce291d95bc8fd48010cecf71"
server_key_iv: "04b7478ea671223305fefcbb1310a69a"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.774698 26764 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:30.785293 26764 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:30.785532 26764 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:30.785648 26764 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:30.785835 26764 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:30.786159 26764 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:30.786201 26764 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:30.786228 26764 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:30.786252 26764 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:30.791627 26764 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:44639
I20260806 03:52:30.791687 26951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:44639 every 8 connection(s)
I20260806 03:52:30.791953 26764 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:52:30.796635 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:30.792576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:49371 (local address 127.12.15.126:46651)
0806 03:52:30.792619 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:30.792621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:30.792858 (+ 237us) server_negotiation.cc:407] Connection header received
0806 03:52:30.793303 (+ 445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:30.793304 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:30.793314 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:30.793368 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:30.793983 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.794489 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.795415 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.795603 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.795647 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.796416 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:30.796417 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:30.796418 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:52:30.796424 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:30.796441 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:52:30.796444 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:30.796487 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:30.796552 (+ 65us) server_negotiation.cc:299] Negotiation successful
0806 03:52:30.796576 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:52:30.796818 26954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:30.792554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:46651 (local address 127.12.15.65:49371)
0806 03:52:30.792747 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:30.792785 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:30.793230 (+ 445us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:30.793421 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:30.793424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:30.793433 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:30.793918 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:30.793922 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.794568 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.794569 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:30.795355 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:30.795357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.795669 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.795670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:30.795772 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.796274 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:52:30.796275 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:30.796302 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:52:30.796326 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:30.796499 (+ 173us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:30.796502 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:30.796504 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:52:30.796548 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:52:30.796624 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":71,"threads_started":1}
I20260806 03:52:30.797291 26952 heartbeater.cc:347] Connected to a master server at 127.12.15.126:46651
I20260806 03:52:30.797369 26952 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:30.797515 26952 heartbeater.cc:511] Master 127.12.15.126:46651 requested a full tablet report, sending...
I20260806 03:52:30.797972 26669 ts_manager.cc:194] Registered new tserver with Master: 08cbae34ed154c9684cbc8a024e130fe (127.12.15.65:44639)
I20260806 03:52:30.798581 26669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.65:49371
I20260806 03:52:30.800892 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 26764
I20260806 03:52:30.800972 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:52:30.801131 12349 external_mini_cluster.cc:1428] Setting key dc47ddbfe40337bf96a5feaa3ae4e55b
I20260806 03:52:30.801926 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:46651
--builtin_ntp_servers=127.12.15.84:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260806 03:52:30.881428 26955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:30.881623 26955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:30.881641 26955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:30.882746 26955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:30.882786 26955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:30.882810 26955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:30.884085 26955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:46651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:30.884294 26955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:30.884557 26955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:52:30.889235 26955 server_base.cc:1034] running on GCE node
W20260806 03:52:30.889218 26972 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:30.889218 26969 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:30.889209 26966 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:30.889645 26955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:30.889869 26955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:30.891006 26955 hybrid_clock.cc:648] HybridClock initialized: now 1785988350890991 us; error 28 us; skew 500 ppm
I20260806 03:52:30.892102 26955 webserver.cc:460] Webserver started at http://127.12.15.66:40831/ using document root <none> and password file <none>
I20260806 03:52:30.892303 26955 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:30.892351 26955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:30.892475 26955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:30.893440 26955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "6caca349d3b94517b6857ce711baed7f"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "35e662990a00c1c80567a16b91202562"
server_key_iv: "8ae2458d0e5600501c30d5a0f64b804f"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.893769 26955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "6caca349d3b94517b6857ce711baed7f"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "35e662990a00c1c80567a16b91202562"
server_key_iv: "8ae2458d0e5600501c30d5a0f64b804f"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.895005 26955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:52:30.895634 26980 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:30.895788 26955 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:30.895867 26955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "6caca349d3b94517b6857ce711baed7f"
format_stamp: "Formatted at 2026-08-06 03:52:30 on dist-test-slave-bqcl"
server_key: "35e662990a00c1c80567a16b91202562"
server_key_iv: "8ae2458d0e5600501c30d5a0f64b804f"
server_key_version: "encryptionkey@0"
I20260806 03:52:30.895927 26955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:30.919646 26955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:30.919940 26955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:30.920079 26955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:30.920297 26955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:30.920632 26955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:30.920674 26955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:30.920704 26955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:30.920722 26955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:30.926165 26955 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:43487
I20260806 03:52:30.926227 27152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:43487 every 8 connection(s)
I20260806 03:52:30.926539 26955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:52:30.926898 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 26955
I20260806 03:52:30.926991 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:52:30.927176 12349 external_mini_cluster.cc:1428] Setting key 1fcc48b3202aebe22f4d8b41bb0a0f48
I20260806 03:52:30.928120 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:46651
--builtin_ntp_servers=127.12.15.84:45117
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260806 03:52:30.932114 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:30.927266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:50651 (local address 127.12.15.126:46651)
0806 03:52:30.927331 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:30.927334 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:30.927583 (+ 249us) server_negotiation.cc:407] Connection header received
0806 03:52:30.928186 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:30.928187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:30.928199 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:30.928260 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:30.929168 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.929671 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.930766 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.931015 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.931072 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.931841 (+ 769us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:30.931842 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:30.931843 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:52:30.931847 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:30.931865 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:52:30.931869 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:30.931916 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:30.931995 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 03:52:30.932025 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 03:52:30.932304 27156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:30.927258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:46651 (local address 127.12.15.66:50651)
0806 03:52:30.927462 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:30.927510 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:30.928081 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:30.928395 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:30.928399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:30.928409 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:30.929071 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:30.929078 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.929799 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.929800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:30.930677 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:30.930680 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:30.931083 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:30.931084 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:30.931195 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:30.931705 (+ 510us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:52:30.931705 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:30.931735 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:52:30.931759 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:30.931923 (+ 164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:30.931928 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:30.931932 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:30.931994 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:52:30.932099 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":84,"threads_started":1}
I20260806 03:52:30.932796 27153 heartbeater.cc:347] Connected to a master server at 127.12.15.126:46651
I20260806 03:52:30.932880 27153 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:30.933050 27153 heartbeater.cc:511] Master 127.12.15.126:46651 requested a full tablet report, sending...
I20260806 03:52:30.933419 26669 ts_manager.cc:194] Registered new tserver with Master: 6caca349d3b94517b6857ce711baed7f (127.12.15.66:43487)
I20260806 03:52:30.933770 26669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.66:50651
W20260806 03:52:31.010469 27159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:31.010676 27159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:31.010703 27159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:31.011833 27159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:31.011873 27159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:31.011906 27159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:31.013219 27159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:45117
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:46651
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:31.013454 27159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:31.013736 27159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:52:31.018707 27159 server_base.cc:1034] running on GCE node
W20260806 03:52:31.018755 27164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.018682 27169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.018680 27165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:31.019099 27159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:31.019330 27159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:31.020458 27159 hybrid_clock.cc:648] HybridClock initialized: now 1785988351020445 us; error 31 us; skew 500 ppm
I20260806 03:52:31.021626 27159 webserver.cc:460] Webserver started at http://127.12.15.67:45425/ using document root <none> and password file <none>
I20260806 03:52:31.021857 27159 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:31.021900 27159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:31.022029 27159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:31.023092 27159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "728c54257aba47128bbd9eb59cc37ef9"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "5f7c8119057216d0b4fd845844fe0631"
server_key_iv: "01056784f4c0d892c7e8d452dc9f14be"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.023422 27159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "728c54257aba47128bbd9eb59cc37ef9"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "5f7c8119057216d0b4fd845844fe0631"
server_key_iv: "01056784f4c0d892c7e8d452dc9f14be"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.024663 27159 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:52:31.025331 27175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.025465 27159 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:31.025549 27159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "728c54257aba47128bbd9eb59cc37ef9"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "5f7c8119057216d0b4fd845844fe0631"
server_key_iv: "01056784f4c0d892c7e8d452dc9f14be"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.025614 27159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:31.043516 27159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:31.043802 27159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:31.043929 27159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:31.044126 27159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:31.044435 27159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:31.044468 27159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.044497 27159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:31.044515 27159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.049865 27159 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:41969
I20260806 03:52:31.049930 27355 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:41969 every 8 connection(s)
I20260806 03:52:31.050307 27159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:52:31.054247 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 27159
I20260806 03:52:31.054342 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:52:31.054593 12349 external_mini_cluster.cc:1428] Setting key 7556ab332f583cfa9ed7ae726ed42c1b
I20260806 03:52:31.055259 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.051006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:45475 (local address 127.12.15.126:46651)
0806 03:52:31.051067 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:31.051069 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:31.051296 (+ 227us) server_negotiation.cc:407] Connection header received
0806 03:52:31.051757 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:31.051758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:31.051767 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:31.051821 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:31.052448 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:31.052933 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:31.053834 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:31.054006 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:31.054047 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:31.054985 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:31.054987 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:31.054987 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:52:31.054992 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:31.055008 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:52:31.055013 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:31.055067 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:31.055151 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 03:52:31.055182 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:52:31.055485 27358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.050976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:46651 (local address 127.12.15.67:45475)
0806 03:52:31.051178 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:31.051218 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:31.051683 (+ 465us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:31.051883 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:31.051886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:31.051895 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:31.052388 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:31.052392 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:31.052995 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:31.052995 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:31.053773 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:31.053775 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:31.054087 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:31.054088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:31.054274 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:31.054841 (+ 567us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:52:31.054842 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:31.054879 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:52:31.054902 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:31.055073 (+ 171us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:31.055080 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:31.055083 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:31.055153 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 03:52:31.055267 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":89,"threads_started":1}
I20260806 03:52:31.056025 27356 heartbeater.cc:347] Connected to a master server at 127.12.15.126:46651
I20260806 03:52:31.056125 27356 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:31.056294 27356 heartbeater.cc:511] Master 127.12.15.126:46651 requested a full tablet report, sending...
I20260806 03:52:31.056629 26669 ts_manager.cc:194] Registered new tserver with Master: 728c54257aba47128bbd9eb59cc37ef9 (127.12.15.67:41969)
I20260806 03:52:31.056950 26669 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.67:45475
I20260806 03:52:31.066188 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 03:52:31.067852 26763 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:31.067433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.12.15.126:46651)
0806 03:52:31.067485 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:31.067487 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:31.067591 (+ 104us) server_negotiation.cc:407] Connection header received
0806 03:52:31.067632 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:31.067632 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:31.067641 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:31.067689 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:31.067791 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33146: BlockingRecv error: recv got EOF from 127.0.0.1:33146 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:52:31.067940 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 26764
I20260806 03:52:31.082634 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 26955
I20260806 03:52:31.088440 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 27159
I20260806 03:52:31.100718 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 26629
2026-08-06T03:52:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (613 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-06T03:52:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:31Z Disabled control of system clock
I20260806 03:52:31.131920 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:39189
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:39189
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 03:52:31.211344 27368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:31.211535 27368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:31.211555 27368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:31.212648 27368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:31.212699 27368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:31.212710 27368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:31.212721 27368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:31.212733 27368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:31.214020 27368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:39189
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:39189
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:31.214263 27368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:31.214524 27368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:31.218973 27375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.218990 27376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.219156 27379 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:31.219235 27368 server_base.cc:1034] running on GCE node
I20260806 03:52:31.219506 27368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:31.219803 27368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:31.220958 27368 hybrid_clock.cc:648] HybridClock initialized: now 1785988351220942 us; error 31 us; skew 500 ppm
I20260806 03:52:31.222153 27368 webserver.cc:460] Webserver started at http://127.12.15.126:39753/ using document root <none> and password file <none>
I20260806 03:52:31.222361 27368 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:31.222414 27368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:31.222568 27368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:31.223598 27368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "dddb2a30049a41568167bebd271cb71e"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "aaef7973a4b27caf5ef3dced5d27a1d0"
server_key_iv: "bc9817d5d0ab83da35d48ff5012e531f"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.223924 27368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "dddb2a30049a41568167bebd271cb71e"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "aaef7973a4b27caf5ef3dced5d27a1d0"
server_key_iv: "bc9817d5d0ab83da35d48ff5012e531f"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.225212 27368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:31.225911 27389 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.226052 27368 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:31.226153 27368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "dddb2a30049a41568167bebd271cb71e"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "aaef7973a4b27caf5ef3dced5d27a1d0"
server_key_iv: "bc9817d5d0ab83da35d48ff5012e531f"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.226220 27368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:31.229321 27368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:31.229580 27368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:31.229698 27368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:31.233230 27368 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:39189
I20260806 03:52:31.233268 27465 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:39189 every 8 connection(s)
I20260806 03:52:31.233544 27368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:52:31.234161 27466 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:31.236003 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 27368
I20260806 03:52:31.236095 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:52:31.236248 12349 external_mini_cluster.cc:1428] Setting key 80c553598e98568574d9f6c7770d8bfa
I20260806 03:52:31.236954 27466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e: Bootstrap starting.
I20260806 03:52:31.237620 27466 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:31.237950 27466 log.cc:824] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:31.238729 27466 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e: No bootstrap required, opened a new log
I20260806 03:52:31.238731 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.236698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48916 (local address 127.12.15.126:39189)
0806 03:52:31.236903 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:31.236912 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:31.236943 (+ 31us) server_negotiation.cc:407] Connection header received
0806 03:52:31.237479 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:31.237488 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:31.237547 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:31.237701 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:31.238292 (+ 591us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:31.238296 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:31.238296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:52:31.238301 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:31.238348 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:52:31.238362 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:31.238447 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:31.238497 (+ 50us) server_negotiation.cc:299] Negotiation successful
0806 03:52:31.238548 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":87,"threads_started":1}
I20260806 03:52:31.240540 27466 raft_consensus.cc:348] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddb2a30049a41568167bebd271cb71e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39189 } }
I20260806 03:52:31.240664 27466 raft_consensus.cc:374] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:31.240684 27466 raft_consensus.cc:729] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dddb2a30049a41568167bebd271cb71e, State: Initialized, Role: FOLLOWER
I20260806 03:52:31.240792 27466 consensus_queue.cc:260] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddb2a30049a41568167bebd271cb71e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39189 } }
I20260806 03:52:31.240855 27466 raft_consensus.cc:388] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:52:31.240880 27466 raft_consensus.cc:482] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:52:31.240926 27466 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:31.241389 27466 raft_consensus.cc:504] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddb2a30049a41568167bebd271cb71e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39189 } }
I20260806 03:52:31.241478 27466 leader_election.cc:302] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dddb2a30049a41568167bebd271cb71e; no voters:
I20260806 03:52:31.241641 27466 leader_election.cc:288] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:52:31.241695 27474 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:31.241847 27474 raft_consensus.cc:686] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [term 1 LEADER]: Becoming Leader. State: Replica: dddb2a30049a41568167bebd271cb71e, State: Running, Role: LEADER
I20260806 03:52:31.241853 27466 sys_catalog.cc:564] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:31.242038 27474 consensus_queue.cc:237] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddb2a30049a41568167bebd271cb71e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39189 } }
I20260806 03:52:31.242443 27475 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dddb2a30049a41568167bebd271cb71e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddb2a30049a41568167bebd271cb71e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39189 } } }
I20260806 03:52:31.242559 27476 sys_catalog.cc:455] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dddb2a30049a41568167bebd271cb71e. Latest consensus state: current_term: 1 leader_uuid: "dddb2a30049a41568167bebd271cb71e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dddb2a30049a41568167bebd271cb71e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39189 } } }
I20260806 03:52:31.242620 27475 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:31.242733 27476 sys_catalog.cc:458] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:31.243052 27484 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:31.243492 27484 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:31.244932 27484 catalog_manager.cc:1347] Generated new cluster ID: d711c558b81a4279abb38b45f6c1990c
I20260806 03:52:31.244989 27484 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:52:31.261998 27484 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:52:31.262218 27484 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:52:31.266415 27484 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dddb2a30049a41568167bebd271cb71e: Generated new TSK 0
I20260806 03:52:31.266618 27484 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:52:31.273447 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39189
--builtin_ntp_servers=127.12.15.84:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 03:52:31.355280 27498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:31.355494 27498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:31.355526 27498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:31.356685 27498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:31.356726 27498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:31.356760 27498 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:31.358084 27498 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:31.358346 27498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:31.358623 27498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:31.363693 27504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.363762 27507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.363940 27505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:31.364110 27498 server_base.cc:1034] running on GCE node
I20260806 03:52:31.364310 27498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:31.364539 27498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:31.365676 27498 hybrid_clock.cc:648] HybridClock initialized: now 1785988351365653 us; error 37 us; skew 500 ppm
I20260806 03:52:31.367110 27498 webserver.cc:460] Webserver started at http://127.12.15.65:38941/ using document root <none> and password file <none>
I20260806 03:52:31.367358 27498 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:31.367416 27498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:31.367550 27498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:31.368690 27498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "7ffd23487ed44e3ab67134ba94d7f309"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "dba670fabe4f15b90104b8896c6d9778"
server_key_iv: "c50a56e3cb693df74fd3008506413aff"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.369027 27498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "7ffd23487ed44e3ab67134ba94d7f309"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "dba670fabe4f15b90104b8896c6d9778"
server_key_iv: "c50a56e3cb693df74fd3008506413aff"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.370393 27498 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:31.371057 27518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.371217 27498 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:31.371301 27498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "7ffd23487ed44e3ab67134ba94d7f309"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "dba670fabe4f15b90104b8896c6d9778"
server_key_iv: "c50a56e3cb693df74fd3008506413aff"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.371363 27498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:31.374545 27498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:31.374780 27498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:31.374897 27498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:31.375085 27498 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:31.375379 27498 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:31.375413 27498 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.375440 27498 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:31.375463 27498 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.380937 27498 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:43461
I20260806 03:52:31.381004 27701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:43461 every 8 connection(s)
I20260806 03:52:31.381273 27498 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:52:31.383842 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.381902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:43199 (local address 127.12.15.126:39189)
0806 03:52:31.381979 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:31.381981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:31.382175 (+ 194us) server_negotiation.cc:407] Connection header received
0806 03:52:31.382616 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:31.382617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:31.382622 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:31.382681 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:31.383671 (+ 990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:31.383672 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:31.383672 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:52:31.383676 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:31.383689 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:52:31.383693 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:31.383735 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:31.383780 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 03:52:31.383790 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 03:52:31.384020 27705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.381874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:39189 (local address 127.12.15.65:43199)
0806 03:52:31.382055 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:31.382086 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:31.382538 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:31.382732 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:31.382735 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:31.382743 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:31.383537 (+ 794us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:52:31.383537 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:31.383563 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:52:31.383589 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:31.383735 (+ 146us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:31.383737 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:31.383739 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:52:31.383787 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 03:52:31.383831 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260806 03:52:31.384473 27702 heartbeater.cc:347] Connected to a master server at 127.12.15.126:39189
I20260806 03:52:31.384558 27702 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:31.384702 27702 heartbeater.cc:511] Master 127.12.15.126:39189 requested a full tablet report, sending...
I20260806 03:52:31.385133 27419 ts_manager.cc:194] Registered new tserver with Master: 7ffd23487ed44e3ab67134ba94d7f309 (127.12.15.65:43461)
I20260806 03:52:31.388831 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 27498
I20260806 03:52:31.388913 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:52:31.389055 12349 external_mini_cluster.cc:1428] Setting key f18c5ad094653f932b2e92a34647bd52
I20260806 03:52:31.389845 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39189
--builtin_ntp_servers=127.12.15.84:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 03:52:31.471413 27706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:31.471612 27706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:31.471633 27706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:31.472743 27706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:31.472775 27706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:31.472800 27706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:31.474195 27706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:31.474371 27706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:31.474613 27706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:31.479313 27711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.479290 27716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.479337 27713 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:31.479395 27706 server_base.cc:1034] running on GCE node
I20260806 03:52:31.479704 27706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:31.479898 27706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:31.481073 27706 hybrid_clock.cc:648] HybridClock initialized: now 1785988351481058 us; error 28 us; skew 500 ppm
I20260806 03:52:31.482270 27706 webserver.cc:460] Webserver started at http://127.12.15.66:45019/ using document root <none> and password file <none>
I20260806 03:52:31.482475 27706 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:31.482518 27706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:31.482648 27706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:31.483636 27706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "201a0007472a4deda6d405fc7d23a56c"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "673ac0795d569cbacf8d64f8ca8171fe"
server_key_iv: "0241622f05443360b2dddd61aa2f0fe6"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.483973 27706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "201a0007472a4deda6d405fc7d23a56c"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "673ac0795d569cbacf8d64f8ca8171fe"
server_key_iv: "0241622f05443360b2dddd61aa2f0fe6"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.485281 27706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:52:31.485946 27727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.486097 27706 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:31.486203 27706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "201a0007472a4deda6d405fc7d23a56c"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "673ac0795d569cbacf8d64f8ca8171fe"
server_key_iv: "0241622f05443360b2dddd61aa2f0fe6"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.486271 27706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:31.489409 27706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:31.489666 27706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:31.489778 27706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:31.489966 27706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:31.490276 27706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:31.490310 27706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.490338 27706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:31.490361 27706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.495689 27706 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:33991
I20260806 03:52:31.495725 27909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:33991 every 8 connection(s)
I20260806 03:52:31.496013 27706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:52:31.498631 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.496664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:44635 (local address 127.12.15.126:39189)
0806 03:52:31.496728 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:31.496730 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:31.496925 (+ 195us) server_negotiation.cc:407] Connection header received
0806 03:52:31.497335 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:31.497335 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:31.497341 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:31.497385 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:31.498444 (+ 1059us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:31.498445 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:31.498445 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:52:31.498449 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:31.498465 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:52:31.498468 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:31.498519 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:31.498566 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:52:31.498577 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:52:31.498795 27913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.496651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:39189 (local address 127.12.15.66:44635)
0806 03:52:31.496817 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:31.496848 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:31.497263 (+ 415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:31.497448 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:31.497451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:31.497459 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:31.498315 (+ 856us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:52:31.498316 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:31.498343 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:52:31.498369 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:31.498517 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:31.498519 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:31.498521 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:52:31.498569 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 03:52:31.498613 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":84,"threads_started":1}
I20260806 03:52:31.499258 27911 heartbeater.cc:347] Connected to a master server at 127.12.15.126:39189
I20260806 03:52:31.499346 27911 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:31.499473 27911 heartbeater.cc:511] Master 127.12.15.126:39189 requested a full tablet report, sending...
I20260806 03:52:31.499794 27419 ts_manager.cc:194] Registered new tserver with Master: 201a0007472a4deda6d405fc7d23a56c (127.12.15.66:33991)
I20260806 03:52:31.504982 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 27706
I20260806 03:52:31.505081 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:52:31.505275 12349 external_mini_cluster.cc:1428] Setting key 4d10ea53777cb690e5a74ed2e0ab5bd4
I20260806 03:52:31.506294 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39189
--builtin_ntp_servers=127.12.15.84:33491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260806 03:52:31.586720 27914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:31.586915 27914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:31.586935 27914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:31.588061 27914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:31.588101 27914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:31.588124 27914 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:31.589475 27914 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39189
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:31.589706 27914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:31.589954 27914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:31.594630 27930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.594656 27923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:31.594631 27927 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:31.594807 27914 server_base.cc:1034] running on GCE node
I20260806 03:52:31.595005 27914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:31.595228 27914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:31.596390 27914 hybrid_clock.cc:648] HybridClock initialized: now 1785988351596372 us; error 33 us; skew 500 ppm
I20260806 03:52:31.597646 27914 webserver.cc:460] Webserver started at http://127.12.15.67:43701/ using document root <none> and password file <none>
I20260806 03:52:31.597872 27914 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:31.597921 27914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:31.598047 27914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:31.599117 27914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "79eff6d2b85c4c72b1f0cd3be20d6c66"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "4b7ee5011ee946bae90c73a3c69b08fe"
server_key_iv: "afaf5e7c1d3e083f5ee8099fe850d4c4"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.599479 27914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "79eff6d2b85c4c72b1f0cd3be20d6c66"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "4b7ee5011ee946bae90c73a3c69b08fe"
server_key_iv: "afaf5e7c1d3e083f5ee8099fe850d4c4"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.600719 27914 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:52:31.601475 27937 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.601646 27914 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:31.601740 27914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "79eff6d2b85c4c72b1f0cd3be20d6c66"
format_stamp: "Formatted at 2026-08-06 03:52:31 on dist-test-slave-bqcl"
server_key: "4b7ee5011ee946bae90c73a3c69b08fe"
server_key_iv: "afaf5e7c1d3e083f5ee8099fe850d4c4"
server_key_version: "encryptionkey@0"
I20260806 03:52:31.601799 27914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:31.605042 27914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:31.605298 27914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:31.605418 27914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:31.605599 27914 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:31.605906 27914 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:31.605948 27914 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.605975 27914 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:31.605999 27914 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:31.611449 27914 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:43067
I20260806 03:52:31.611485 28104 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:43067 every 8 connection(s)
I20260806 03:52:31.611779 27914 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:52:31.614406 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.612423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:43899 (local address 127.12.15.126:39189)
0806 03:52:31.612508 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:31.612510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:31.612690 (+ 180us) server_negotiation.cc:407] Connection header received
0806 03:52:31.613096 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:31.613097 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:31.613101 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:31.613146 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:31.614223 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:31.614224 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:31.614224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:52:31.614229 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:31.614243 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:52:31.614247 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:31.614289 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:31.614341 (+ 52us) server_negotiation.cc:299] Negotiation successful
0806 03:52:31.614353 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 03:52:31.614586 28107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:31.612396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:39189 (local address 127.12.15.67:43899)
0806 03:52:31.612582 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:31.612613 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:31.613025 (+ 412us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:31.613204 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:31.613207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:31.613214 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:31.614040 (+ 826us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:52:31.614041 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:31.614067 (+ 26us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:52:31.614095 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:31.614291 (+ 196us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:31.614294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:31.614297 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:31.614349 (+ 52us) client_negotiation.cc:240] Negotiation successful
0806 03:52:31.614396 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":82,"threads_started":1}
I20260806 03:52:31.615005 28105 heartbeater.cc:347] Connected to a master server at 127.12.15.126:39189
I20260806 03:52:31.615090 28105 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:31.615203 28105 heartbeater.cc:511] Master 127.12.15.126:39189 requested a full tablet report, sending...
I20260806 03:52:31.615517 27419 ts_manager.cc:194] Registered new tserver with Master: 79eff6d2b85c4c72b1f0cd3be20d6c66 (127.12.15.67:43067)
I20260806 03:52:31.621315 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 27914
I20260806 03:52:31.621398 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:52:31.621587 12349 external_mini_cluster.cc:1428] Setting key 6154cf2b34c36c90c3265989ecb122d4
I20260806 03:52:31.622633 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260806 03:52:31.624480 27470 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:52:31.623952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48926 (local address 127.12.15.126:39189)
0806 03:52:31.624009 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:31.624011 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:31.624126 (+ 115us) server_negotiation.cc:407] Connection header received
0806 03:52:31.624204 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:31.624205 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:31.624210 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:31.624255 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:31.624406 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48926: BlockingRecv error: recv got EOF from 127.0.0.1:48926 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:52:31.624522 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 27498
I20260806 03:52:31.645608 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 27706
I20260806 03:52:31.659539 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 27914
I20260806 03:52:31.668849 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 27368
2026-08-06T03:52:31Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (568 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:52:31 dist-test-slave-bqcl krb5kdc[28123](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:52:31 dist-test-slave-bqcl krb5kdc[28123](info): set up 2 sockets
Aug 06 03:52:31 dist-test-slave-bqcl krb5kdc[28123](info): commencing operation
krb5kdc: starting...
W20260806 03:52:32.803483 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:52:32 dist-test-slave-bqcl krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:52:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:52:32.919111 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:41793
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:41793
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:32.998494 28144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:32.998703 28144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:32.998734 28144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:33.000028 28144 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:52:33.000087 28144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:33.000109 28144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:33.000126 28144 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:52:33.000140 28144 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:52:33.001685 28144 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:41793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:41793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:33.001936 28144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:33.002174 28144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:33.006764 28155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:33.006780 28153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:33.006767 28151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:33.006997 28144 server_base.cc:1034] running on GCE node
I20260806 03:52:33.007290 28144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:33.007550 28144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:33.008695 28144 hybrid_clock.cc:648] HybridClock initialized: now 1785988353008681 us; error 37 us; skew 500 ppm
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988353, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:33.011091 28144 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:52:33.011472 28144 webserver.cc:460] Webserver started at http://127.12.15.126:35035/ using document root <none> and password file <none>
I20260806 03:52:33.011684 28144 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:33.011732 28144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:33.011858 28144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:33.012830 28144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "4e581c8a30dc4c30a0efb0c0e042133e"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "3e40823c458d45b02beb534fe2e533c3"
server_key_iv: "b0c2d0200f1c1763bf0a0bf80a622529"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.013195 28144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "4e581c8a30dc4c30a0efb0c0e042133e"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "3e40823c458d45b02beb534fe2e533c3"
server_key_iv: "b0c2d0200f1c1763bf0a0bf80a622529"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.014438 28144 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:52:33.015086 28165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.015226 28144 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.015309 28144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "4e581c8a30dc4c30a0efb0c0e042133e"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "3e40823c458d45b02beb534fe2e533c3"
server_key_iv: "b0c2d0200f1c1763bf0a0bf80a622529"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.015369 28144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:33.024752 28144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:33.026733 28144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:33.026861 28144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:33.030576 28144 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:41793
I20260806 03:52:33.030609 28250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:41793 every 8 connection(s)
I20260806 03:52:33.030915 28144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:52:33.031832 28252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:33.033262 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 28144
I20260806 03:52:33.033352 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:52:33.033515 12349 external_mini_cluster.cc:1428] Setting key 146aa8166fa76f9a01c17965c8cf19e9
I20260806 03:52:33.034080 28252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e: Bootstrap starting.
I20260806 03:52:33.034744 28252 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:33.035025 28252 log.cc:824] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:33.035642 28252 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e: No bootstrap required, opened a new log
I20260806 03:52:33.037299 28252 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e581c8a30dc4c30a0efb0c0e042133e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 41793 } }
I20260806 03:52:33.037426 28252 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:33.037458 28252 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e581c8a30dc4c30a0efb0c0e042133e, State: Initialized, Role: FOLLOWER
I20260806 03:52:33.037600 28252 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e581c8a30dc4c30a0efb0c0e042133e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 41793 } }
I20260806 03:52:33.037707 28252 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:52:33.037753 28252 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:52:33.037801 28252 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:33.038501 28252 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e581c8a30dc4c30a0efb0c0e042133e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 41793 } }
I20260806 03:52:33.038614 28252 leader_election.cc:302] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4e581c8a30dc4c30a0efb0c0e042133e; no voters:
I20260806 03:52:33.038774 28252 leader_election.cc:288] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:52:33.038815 28264 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:33.039013 28264 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [term 1 LEADER]: Becoming Leader. State: Replica: 4e581c8a30dc4c30a0efb0c0e042133e, State: Running, Role: LEADER
I20260806 03:52:33.039021 28252 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:52:33.039171 28264 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e581c8a30dc4c30a0efb0c0e042133e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 41793 } }
I20260806 03:52:33.039502 28266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4e581c8a30dc4c30a0efb0c0e042133e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e581c8a30dc4c30a0efb0c0e042133e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 41793 } } }
I20260806 03:52:33.039577 28266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [sys.catalog]: This master's current role is: LEADER
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988352, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:33.039825 28267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4e581c8a30dc4c30a0efb0c0e042133e. Latest consensus state: current_term: 1 leader_uuid: "4e581c8a30dc4c30a0efb0c0e042133e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4e581c8a30dc4c30a0efb0c0e042133e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 41793 } } }
I20260806 03:52:33.039943 28267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e [sys.catalog]: This master's current role is: LEADER
I20260806 03:52:33.040335 28272 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:52:33.040838 28272 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:52:33.042177 28272 catalog_manager.cc:1347] Generated new cluster ID: 1913377824fe4ba6bebd74419184774d
I20260806 03:52:33.042228 28272 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:52:33.043763 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.033988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.12.15.126:41793)
0806 03:52:33.034194 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.034201 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.034233 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:52:33.034649 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.034666 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.034711 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.034836 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:33.035480 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.036350 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.037075 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.037468 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.037565 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.040268 (+ 2703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:33.040272 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:33.040272 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:33.040283 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:33.042212 (+ 1929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.042606 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.042609 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.042610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.042709 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.042918 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.042918 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.042919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.043178 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:33.043292 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.043432 (+ 140us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.043563 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":58,"threads_started":1}
I20260806 03:52:33.065259 28272 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:52:33.066061 28272 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:52:33.073534 28272 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e: Generated new TSK 0
I20260806 03:52:33.073726 28272 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:52:33.101176 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:41793
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:33.183755 28295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:33.183959 28295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:33.183980 28295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:33.185073 28295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:33.185108 28295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:33.185132 28295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:52:33.186636 28295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:41793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:33.186821 28295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:33.187008 28295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:33.191728 28304 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:33.191768 28301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:33.191740 28306 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:33.191886 28295 server_base.cc:1034] running on GCE node
I20260806 03:52:33.192188 28295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:33.192430 28295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:33.193578 28295 hybrid_clock.cc:648] HybridClock initialized: now 1785988353193577 us; error 44 us; skew 500 ppm
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988353, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:33.196162 28295 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:52:33.196591 28295 webserver.cc:460] Webserver started at http://127.12.15.65:36129/ using document root <none> and password file <none>
I20260806 03:52:33.196803 28295 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:33.196851 28295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:33.196978 28295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:33.198000 28295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "046cfb58f5dd4ff3a0a8f964e5192425"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "a36774a2c992bd98d3f5a1e1eb812eff"
server_key_iv: "f790ff80a2cfd0f27867b98e5c928aaa"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.198364 28295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "046cfb58f5dd4ff3a0a8f964e5192425"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "a36774a2c992bd98d3f5a1e1eb812eff"
server_key_iv: "f790ff80a2cfd0f27867b98e5c928aaa"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.199760 28295 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:52:33.200415 28321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.200601 28295 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:33.200677 28295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "046cfb58f5dd4ff3a0a8f964e5192425"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "a36774a2c992bd98d3f5a1e1eb812eff"
server_key_iv: "f790ff80a2cfd0f27867b98e5c928aaa"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.200742 28295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:33.208722 28295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:33.210448 28295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:33.210556 28295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:33.210722 28295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:33.210955 28295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:33.210985 28295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.211004 28295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:33.211021 28295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.216486 28295 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:45959
I20260806 03:52:33.216539 28498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:45959 every 8 connection(s)
I20260806 03:52:33.216850 28295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988353, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:33.225458 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.217552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:54549 (local address 127.12.15.126:41793)
0806 03:52:33.217611 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.217612 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.217811 (+ 199us) server_negotiation.cc:407] Connection header received
0806 03:52:33.218346 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.218346 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.218356 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.218414 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:33.219373 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.219882 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.220864 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.221057 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.221106 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.223028 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:33.223029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:33.223030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:33.223037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:33.224322 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.224717 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.224717 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.224717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.224767 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.224954 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.224955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.224955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.225049 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:33.225093 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.225321 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.225391 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:52:33.225603 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.217523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:41793 (local address 127.12.15.65:54549)
0806 03:52:33.217684 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.217725 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:33.218259 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:33.218473 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:33.218476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:33.219074 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:33.219262 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.219267 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.219984 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.219985 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.220791 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.220793 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.221115 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.221115 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.221209 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.221641 (+ 432us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:33.221648 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:33.222950 (+ 1302us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:33.224404 (+ 1454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:33.224411 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:33.224413 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:33.224655 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:33.224812 (+ 157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:33.224813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:33.224813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:33.224913 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:33.225093 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:33.225095 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:33.225202 (+ 107us) client_negotiation.cc:769] Sending connection context
0806 03:52:33.225293 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 03:52:33.225413 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":82,"threads_started":1}
I20260806 03:52:33.226061 28499 heartbeater.cc:347] Connected to a master server at 127.12.15.126:41793
I20260806 03:52:33.226186 28499 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:33.226262 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 28295
I20260806 03:52:33.226364 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:52:33.226370 28499 heartbeater.cc:511] Master 127.12.15.126:41793 requested a full tablet report, sending...
I20260806 03:52:33.226559 12349 external_mini_cluster.cc:1428] Setting key 894d5e88e3b897b2f9df8bcbc1ab04d5
I20260806 03:52:33.226907 28199 ts_manager.cc:194] Registered new tserver with Master: 046cfb58f5dd4ff3a0a8f964e5192425 (127.12.15.65:45959)
I20260806 03:52:33.227639 28199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:54549
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:52:33.252775 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:41793
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:33.332922 28505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:33.333142 28505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:33.333174 28505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:33.334303 28505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:33.334342 28505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:33.334378 28505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:52:33.335806 28505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:41793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:33.336050 28505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:33.336256 28505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:33.341085 28511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:33.341094 28515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:33.341101 28512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:33.341289 28505 server_base.cc:1034] running on GCE node
I20260806 03:52:33.341516 28505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:33.341743 28505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:33.342887 28505 hybrid_clock.cc:648] HybridClock initialized: now 1785988353342871 us; error 27 us; skew 500 ppm
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988353, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:33.345341 28505 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:52:33.345736 28505 webserver.cc:460] Webserver started at http://127.12.15.66:37711/ using document root <none> and password file <none>
I20260806 03:52:33.345947 28505 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:33.345994 28505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:33.346145 28505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:33.347194 28505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "858a4bc68f2347649ba7ad09339f1385"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "90f91ed14f6f2ddd028fb8c79d6e5c6d"
server_key_iv: "fcbf212e37006b6b7a9f34c9fea09a81"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.347524 28505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "858a4bc68f2347649ba7ad09339f1385"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "90f91ed14f6f2ddd028fb8c79d6e5c6d"
server_key_iv: "fcbf212e37006b6b7a9f34c9fea09a81"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.348737 28505 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:52:33.349391 28523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.349537 28505 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:52:33.349620 28505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "858a4bc68f2347649ba7ad09339f1385"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "90f91ed14f6f2ddd028fb8c79d6e5c6d"
server_key_iv: "fcbf212e37006b6b7a9f34c9fea09a81"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.349681 28505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:33.358369 28505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:33.360526 28505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:33.360661 28505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:33.360849 28505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:33.361137 28505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:33.361171 28505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.361198 28505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:33.361223 28505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.366662 28505 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:42805
I20260806 03:52:33.366717 28704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:42805 every 8 connection(s)
I20260806 03:52:33.366987 28505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:52:33.367311 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 28505
I20260806 03:52:33.367411 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:52:33.367730 12349 external_mini_cluster.cc:1428] Setting key bad334fb654507f728a592edb7447647
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988353, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:33.376082 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.367748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:49113 (local address 127.12.15.126:41793)
0806 03:52:33.367812 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.367815 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.368040 (+ 225us) server_negotiation.cc:407] Connection header received
0806 03:52:33.368630 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.368631 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.368641 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.368682 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:33.369810 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.370340 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.371306 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.371493 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.371548 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.373513 (+ 1965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:33.373515 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:33.373515 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:33.373522 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:33.374820 (+ 1298us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.375197 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.375198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.375198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.375251 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.375478 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.375478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.375479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.375589 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:33.375646 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.375907 (+ 261us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.375991 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
I20260806 03:52:33.376292 28708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.367651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:41793 (local address 127.12.15.66:49113)
0806 03:52:33.367896 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.367945 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:33.368535 (+ 590us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:33.368751 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:33.368755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:33.369471 (+ 716us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:33.369731 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.369738 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.370434 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.370435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.371225 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.371228 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.371561 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.371561 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.371690 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.372116 (+ 426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:33.372124 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:33.373427 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:33.374894 (+ 1467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:33.374901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:33.374904 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:33.375121 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:33.375306 (+ 185us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:33.375307 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:33.375308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:33.375420 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:33.375644 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:33.375659 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:33.375789 (+ 130us) client_negotiation.cc:769] Sending connection context
0806 03:52:33.375876 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:52:33.376031 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":98,"threads_started":1}
I20260806 03:52:33.376745 28705 heartbeater.cc:347] Connected to a master server at 127.12.15.126:41793
I20260806 03:52:33.376834 28705 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:33.377003 28705 heartbeater.cc:511] Master 127.12.15.126:41793 requested a full tablet report, sending...
I20260806 03:52:33.377403 28199 ts_manager.cc:194] Registered new tserver with Master: 858a4bc68f2347649ba7ad09339f1385 (127.12.15.66:42805)
I20260806 03:52:33.377758 28199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:49113
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:52:33.393218 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:41793
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:52:33.474017 28712 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:52:33.474260 28712 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:52:33.474290 28712 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:52:33.475404 28712 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:52:33.475446 28712 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:52:33.475481 28712 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:52:33.476876 28712 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:41793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28712
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:52:33.477113 28712 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:52:33.477332 28712 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:52:33.482348 28719 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:33.482414 28712 server_base.cc:1034] running on GCE node
W20260806 03:52:33.482358 28721 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:52:33.482358 28718 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:52:33.482695 28712 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:52:33.482890 28712 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:52:33.484027 28712 hybrid_clock.cc:648] HybridClock initialized: now 1785988353484014 us; error 31 us; skew 500 ppm
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28123](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988353, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:52:33.486392 28712 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:52:33.486801 28712 webserver.cc:460] Webserver started at http://127.12.15.67:34263/ using document root <none> and password file <none>
I20260806 03:52:33.486985 28712 fs_manager.cc:359] Metadata directory not provided
I20260806 03:52:33.487030 28712 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:52:33.487154 28712 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:52:33.488137 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "e504150cd3cb4109b93e6ca820a732ee"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "89cb8e778d4383ed20c80b7c77a90af7"
server_key_iv: "1ae2190de8916d38338f4508dabebcab"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.488448 28712 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "e504150cd3cb4109b93e6ca820a732ee"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "89cb8e778d4383ed20c80b7c77a90af7"
server_key_iv: "1ae2190de8916d38338f4508dabebcab"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.489681 28712 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:52:33.490403 28737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.490549 28712 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:52:33.490634 28712 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "e504150cd3cb4109b93e6ca820a732ee"
format_stamp: "Formatted at 2026-08-06 03:52:33 on dist-test-slave-bqcl"
server_key: "89cb8e778d4383ed20c80b7c77a90af7"
server_key_iv: "1ae2190de8916d38338f4508dabebcab"
server_key_version: "encryptionkey@0"
I20260806 03:52:33.490694 28712 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:52:33.505463 28712 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:52:33.507484 28712 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:52:33.507623 28712 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:52:33.507813 28712 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:52:33.508092 28712 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:52:33.508126 28712 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.508152 28712 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:52:33.508176 28712 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:52:33.513706 28712 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:38971
I20260806 03:52:33.513759 28917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:38971 every 8 connection(s)
I20260806 03:52:33.514043 28712 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:52:33.518236 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 28712
I20260806 03:52:33.518328 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:52:33.518503 12349 external_mini_cluster.cc:1428] Setting key a3e1a45da769a9c70ae221565d8320dd
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28123](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988353, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:52:33.522511 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.514730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:54527 (local address 127.12.15.126:41793)
0806 03:52:33.514804 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.514806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.514979 (+ 173us) server_negotiation.cc:407] Connection header received
0806 03:52:33.515447 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.515448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.515458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.515506 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:33.516354 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.516848 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.517824 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.518017 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.518075 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.520195 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:33.520196 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:33.520197 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:33.520203 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:33.521390 (+ 1187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.521738 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.521739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.521739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.521784 (+ 45us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.521955 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.521955 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.521955 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.522046 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:33.522089 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.522379 (+ 290us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.522449 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:52:33.522701 28921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.514700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:41793 (local address 127.12.15.67:54527)
0806 03:52:33.514871 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.514905 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:33.515371 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:33.515569 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:33.515573 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:33.516109 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:52:33.516290 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.516294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.516926 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.516927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.517762 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.517765 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.518088 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.518089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.518206 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.518641 (+ 435us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:52:33.518650 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:52:33.520113 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:52:33.521456 (+ 1343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:33.521463 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:33.521465 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:33.521683 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:33.521827 (+ 144us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:52:33.521827 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:52:33.521827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:52:33.521915 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:52:33.522085 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:52:33.522088 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:52:33.522245 (+ 157us) client_negotiation.cc:769] Sending connection context
0806 03:52:33.522351 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 03:52:33.522472 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":100,"threads_started":1}
I20260806 03:52:33.523130 28918 heartbeater.cc:347] Connected to a master server at 127.12.15.126:41793
I20260806 03:52:33.523217 28918 heartbeater.cc:464] Registering TS with master...
I20260806 03:52:33.523350 28918 heartbeater.cc:511] Master 127.12.15.126:41793 requested a full tablet report, sending...
I20260806 03:52:33.523699 28199 ts_manager.cc:194] Registered new tserver with Master: e504150cd3cb4109b93e6ca820a732ee (127.12.15.67:38971)
I20260806 03:52:33.524021 28199 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:54527
I20260806 03:52:33.530030 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:52:33.536516 28258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.531359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51106 (local address 127.12.15.126:41793)
0806 03:52:33.531409 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.531410 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.531519 (+ 109us) server_negotiation.cc:407] Connection header received
0806 03:52:33.531560 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.531561 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.531569 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.531610 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:52:33.532104 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.532550 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.533205 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.533369 (+ 164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.533407 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.534561 (+ 1154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:52:33.534561 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:52:33.534562 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:52:33.534566 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:52:33.535710 (+ 1144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.535988 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.535989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.535989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.536030 (+ 41us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:52:33.536141 (+ 111us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:52:33.536141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:52:33.536141 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:52:33.536256 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:52:33.536325 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.536399 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.536457 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:52:33.537911 28199 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51106:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:52:33.539577 28199 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:52:33.545079 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.541577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:45959 (local address 127.0.0.1:52474)
0806 03:52:33.541802 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.541845 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:33.541909 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:33.542225 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:33.542228 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:33.542231 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:33.542292 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.542293 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.543175 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.543176 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.544076 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.544079 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.544803 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.544804 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.544883 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.544897 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:52:33.544955 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:52:33.545008 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":48,"threads_started":1}
I20260806 03:52:33.545099 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.541700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52474 (local address 127.12.15.65:45959)
0806 03:52:33.541873 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.541879 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.541889 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:33.541989 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.541992 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.542006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.542078 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:33.542372 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.543079 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.544153 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.544726 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.544791 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.544851 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.544978 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.545026 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":88,"threads_started":1}
I20260806 03:52:33.545639 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.541582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38971 (local address 127.0.0.1:59688)
0806 03:52:33.541875 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.541885 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:33.541938 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:33.542345 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:33.542346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:33.542348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:33.542400 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.542402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.543708 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.543709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.544521 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.544524 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.545408 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.545409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.545471 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.545476 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:52:33.545532 (+ 56us) client_negotiation.cc:240] Negotiation successful
0806 03:52:33.545571 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":13,"thread_start_us":51,"threads_started":1}
I20260806 03:52:33.545658 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.541790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59688 (local address 127.12.15.67:38971)
0806 03:52:33.542190 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.542194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.542203 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:52:33.542232 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.542234 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.542249 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.542469 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:33.542521 (+ 52us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.543573 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.544592 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.545323 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.545395 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.545445 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.545549 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.545582 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":70,"threads_started":1}
I20260806 03:52:33.546159 28400 tablet_service.cc:1467] Processing CreateTablet for tablet 114acdeeee0a4c22aeb2d4c89e267add (DEFAULT_TABLE table=test-table [id=2a1968be22d3435e8cb7ff37ab429118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:33.546535 28400 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114acdeeee0a4c22aeb2d4c89e267add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:33.546769 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.541583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:42805 (local address 127.0.0.1:59184)
0806 03:52:33.542018 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.542028 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:33.542063 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:33.542379 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:33.542379 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:33.542381 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:33.542408 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.542409 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.543708 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.543709 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.545651 (+ 1942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.545664 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.546537 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.546538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.546599 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.546603 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:52:33.546650 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:52:33.546701 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":4,"spinlock_wait_cycles":1280,"thread_start_us":50,"threads_started":1}
I20260806 03:52:33.546808 28812 tablet_service.cc:1467] Processing CreateTablet for tablet 114acdeeee0a4c22aeb2d4c89e267add (DEFAULT_TABLE table=test-table [id=2a1968be22d3435e8cb7ff37ab429118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:33.547063 28939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.541954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59184 (local address 127.12.15.66:42805)
0806 03:52:33.542146 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.542151 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.542162 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:52:33.542203 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.542205 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.542221 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.542302 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:33.542480 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.543590 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.545760 (+ 2170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.546382 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.546446 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.546502 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.546945 (+ 443us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.546990 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":87,"threads_started":1}
I20260806 03:52:33.547161 28812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114acdeeee0a4c22aeb2d4c89e267add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:33.547734 28599 tablet_service.cc:1467] Processing CreateTablet for tablet 114acdeeee0a4c22aeb2d4c89e267add (DEFAULT_TABLE table=test-table [id=2a1968be22d3435e8cb7ff37ab429118]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:52:33.548045 28599 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 114acdeeee0a4c22aeb2d4c89e267add. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:52:33.549296 28940 tablet_bootstrap.cc:492] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425: Bootstrap starting.
I20260806 03:52:33.549834 28940 tablet_bootstrap.cc:654] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:33.550333 28940 log.cc:824] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:33.550583 28941 tablet_bootstrap.cc:492] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: Bootstrap starting.
I20260806 03:52:33.551190 28941 tablet_bootstrap.cc:654] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:33.551195 28940 tablet_bootstrap.cc:492] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425: No bootstrap required, opened a new log
I20260806 03:52:33.551258 28940 ts_tablet_manager.cc:1397] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:33.551317 28943 tablet_bootstrap.cc:492] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Bootstrap starting.
I20260806 03:52:33.551781 28941 log.cc:824] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:33.551962 28943 tablet_bootstrap.cc:654] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Neither blocks nor log segments found. Creating new log.
I20260806 03:52:33.552542 28943 log.cc:824] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Log is configured to *not* fsync() on all Append() calls
I20260806 03:52:33.552573 28941 tablet_bootstrap.cc:492] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: No bootstrap required, opened a new log
I20260806 03:52:33.552634 28941 ts_tablet_manager.cc:1397] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:33.552940 28940 raft_consensus.cc:348] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.553097 28940 raft_consensus.cc:374] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:33.553148 28940 raft_consensus.cc:729] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 046cfb58f5dd4ff3a0a8f964e5192425, State: Initialized, Role: FOLLOWER
I20260806 03:52:33.553294 28943 tablet_bootstrap.cc:492] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: No bootstrap required, opened a new log
I20260806 03:52:33.553274 28940 consensus_queue.cc:260] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.553362 28943 ts_tablet_manager.cc:1397] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:52:33.553541 28940 ts_tablet_manager.cc:1428] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260806 03:52:33.553669 28499 heartbeater.cc:503] Master 127.12.15.126:41793 was elected leader, sending a full tablet report...
I20260806 03:52:33.554968 28941 raft_consensus.cc:348] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.555142 28941 raft_consensus.cc:374] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:33.555178 28941 raft_consensus.cc:729] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e504150cd3cb4109b93e6ca820a732ee, State: Initialized, Role: FOLLOWER
I20260806 03:52:33.555284 28941 consensus_queue.cc:260] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.555225 28943 raft_consensus.cc:348] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.555357 28943 raft_consensus.cc:374] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:52:33.555402 28943 raft_consensus.cc:729] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 858a4bc68f2347649ba7ad09339f1385, State: Initialized, Role: FOLLOWER
I20260806 03:52:33.555538 28941 ts_tablet_manager.cc:1428] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:52:33.555516 28943 consensus_queue.cc:260] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.555569 28918 heartbeater.cc:503] Master 127.12.15.126:41793 was elected leader, sending a full tablet report...
I20260806 03:52:33.555755 28943 ts_tablet_manager.cc:1428] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:52:33.555791 28705 heartbeater.cc:503] Master 127.12.15.126:41793 was elected leader, sending a full tablet report...
W20260806 03:52:33.617597 28706 tablet.cc:2369] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:52:33.653745 28947 raft_consensus.cc:482] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:52:33.653867 28947 raft_consensus.cc:504] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.654233 28947 leader_election.cc:288] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e504150cd3cb4109b93e6ca820a732ee (127.12.15.67:38971), 858a4bc68f2347649ba7ad09339f1385 (127.12.15.66:42805)
I20260806 03:52:33.657070 28939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.654488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:56125 (local address 127.12.15.66:42805)
0806 03:52:33.654551 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.654554 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.654619 (+ 65us) server_negotiation.cc:407] Connection header received
0806 03:52:33.654689 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.654690 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.654701 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.654767 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:33.654928 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.655433 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.656377 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.656754 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.656807 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.656815 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.656963 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.657000 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:52:33.657073 28502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.654426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:42805 (local address 127.12.15.65:56125)
0806 03:52:33.654550 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.654568 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:33.654629 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:33.654778 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:33.654780 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:33.654782 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:33.654865 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.654867 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.655506 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.655507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.656293 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.656295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.656828 (+ 533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.656829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.656912 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.656915 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:33.656958 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:52:33.656998 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103}
I20260806 03:52:33.657140 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.654506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:38925 (local address 127.12.15.67:38971)
0806 03:52:33.654551 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.654553 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.654781 (+ 228us) server_negotiation.cc:407] Connection header received
0806 03:52:33.654814 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.654815 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.654826 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.654882 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:52:33.655026 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.655519 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.656413 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.656797 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.656868 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.656876 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.657036 (+ 160us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.657083 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:52:33.657352 28950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.654430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38971 (local address 127.12.15.65:38925)
0806 03:52:33.654702 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.654713 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:52:33.654771 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:52:33.654887 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:52:33.654888 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:52:33.654889 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:52:33.654965 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.654966 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.655597 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.655598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.656324 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:52:33.656325 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.656883 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.656884 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:52:33.656963 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.656966 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:52:33.657017 (+ 51us) client_negotiation.cc:240] Negotiation successful
0806 03:52:33.657055 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":36,"thread_start_us":130,"threads_started":1}
I20260806 03:52:33.657449 28626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114acdeeee0a4c22aeb2d4c89e267add" candidate_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858a4bc68f2347649ba7ad09339f1385" is_pre_election: true
I20260806 03:52:33.657629 28626 raft_consensus.cc:2454] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 046cfb58f5dd4ff3a0a8f964e5192425 in term 0.
I20260806 03:52:33.657719 28842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114acdeeee0a4c22aeb2d4c89e267add" candidate_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e504150cd3cb4109b93e6ca820a732ee" is_pre_election: true
I20260806 03:52:33.657902 28842 raft_consensus.cc:2454] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 046cfb58f5dd4ff3a0a8f964e5192425 in term 0.
I20260806 03:52:33.657883 28327 leader_election.cc:302] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 046cfb58f5dd4ff3a0a8f964e5192425, 858a4bc68f2347649ba7ad09339f1385; no voters:
I20260806 03:52:33.658041 28947 raft_consensus.cc:2781] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:52:33.658092 28947 raft_consensus.cc:482] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:52:33.658107 28947 raft_consensus.cc:3037] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:33.658813 28947 raft_consensus.cc:504] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.658936 28947 leader_election.cc:288] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [CANDIDATE]: Term 1 election: Requested vote from peers e504150cd3cb4109b93e6ca820a732ee (127.12.15.67:38971), 858a4bc68f2347649ba7ad09339f1385 (127.12.15.66:42805)
I20260806 03:52:33.659083 28842 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114acdeeee0a4c22aeb2d4c89e267add" candidate_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e504150cd3cb4109b93e6ca820a732ee"
I20260806 03:52:33.659083 28626 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "114acdeeee0a4c22aeb2d4c89e267add" candidate_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "858a4bc68f2347649ba7ad09339f1385"
I20260806 03:52:33.659157 28626 raft_consensus.cc:3037] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:33.659157 28842 raft_consensus.cc:3037] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:52:33.660028 28626 raft_consensus.cc:2454] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 046cfb58f5dd4ff3a0a8f964e5192425 in term 1.
I20260806 03:52:33.660028 28842 raft_consensus.cc:2454] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 046cfb58f5dd4ff3a0a8f964e5192425 in term 1.
I20260806 03:52:33.660192 28327 leader_election.cc:302] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 046cfb58f5dd4ff3a0a8f964e5192425, 858a4bc68f2347649ba7ad09339f1385; no voters:
I20260806 03:52:33.660295 28947 raft_consensus.cc:2781] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:52:33.660454 28947 raft_consensus.cc:686] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 1 LEADER]: Becoming Leader. State: Replica: 046cfb58f5dd4ff3a0a8f964e5192425, State: Running, Role: LEADER
I20260806 03:52:33.660547 28947 consensus_queue.cc:237] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } }
I20260806 03:52:33.661320 28199 catalog_manager.cc:5495] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 reported cstate change: term changed from 0 to 1, leader changed from <none> to 046cfb58f5dd4ff3a0a8f964e5192425 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "046cfb58f5dd4ff3a0a8f964e5192425" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 45959 } health_report { overall_health: HEALTHY } } }
I20260806 03:52:33.692517 28934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.690069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52480 (local address 127.12.15.65:45959)
0806 03:52:33.690159 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 03:52:33.690161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:52:33.690171 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:52:33.690211 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:52:33.690212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:52:33.690233 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:52:33.690299 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:52:33.690457 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.690937 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.691729 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:52:33.691931 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:52:33.691997 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:52:33.692167 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:52:33.692271 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:52:33.692352 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:52:33.692392 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 03:52:33.692414 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:52:33.694299 28842 raft_consensus.cc:1261] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 1 FOLLOWER]: Refusing update from remote peer 046cfb58f5dd4ff3a0a8f964e5192425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:33.694299 28626 raft_consensus.cc:1261] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 1 FOLLOWER]: Refusing update from remote peer 046cfb58f5dd4ff3a0a8f964e5192425: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:52:33.694630 28947 consensus_queue.cc:1035] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e504150cd3cb4109b93e6ca820a732ee" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38971 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:33.694787 28952 consensus_queue.cc:1035] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [LEADER]: Connected to new peer: Peer: permanent_uuid: "858a4bc68f2347649ba7ad09339f1385" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 42805 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:52:33.697126 28956 mvcc.cc:205] Tried to move back new op lower bound from 7315408296729178112 to 7315408296594173952. Current Snapshot: MvccSnapshot[applied={T|T < 7315408296729178112}]
I20260806 03:52:33.698112 28958 mvcc.cc:205] Tried to move back new op lower bound from 7315408296729178112 to 7315408296594173952. Current Snapshot: MvccSnapshot[applied={T|T < 7315408296729178112 or (T in {7315408296729178112})}]
I20260806 03:52:33.699712 28198 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:33.699807 28198 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51106:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:52:33.700524 28198 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4e581c8a30dc4c30a0efb0c0e042133e: Sending DeleteTablet for 3 replicas of tablet 114acdeeee0a4c22aeb2d4c89e267add
I20260806 03:52:33.700862 28400 tablet_service.cc:1514] Processing DeleteTablet for tablet 114acdeeee0a4c22aeb2d4c89e267add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:33 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:52474
I20260806 03:52:33.700891 28812 tablet_service.cc:1514] Processing DeleteTablet for tablet 114acdeeee0a4c22aeb2d4c89e267add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:33 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:59688
I20260806 03:52:33.701061 28974 tablet_replica.cc:330] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425: stopping tablet replica
I20260806 03:52:33.701110 28599 tablet_service.cc:1514] Processing DeleteTablet for tablet 114acdeeee0a4c22aeb2d4c89e267add with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:52:33 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:59184
I20260806 03:52:33.701166 28974 raft_consensus.cc:2229] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:52:33.701262 28974 raft_consensus.cc:2258] T 114acdeeee0a4c22aeb2d4c89e267add P 046cfb58f5dd4ff3a0a8f964e5192425 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:33.701363 28978 tablet_replica.cc:330] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: stopping tablet replica
I20260806 03:52:33.701480 28977 tablet_replica.cc:330] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: stopping tablet replica
I20260806 03:52:33.701509 28978 raft_consensus.cc:2229] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:33.701560 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 28295
I20260806 03:52:33.701586 28977 raft_consensus.cc:2229] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:52:33.701694 28977 raft_consensus.cc:2258] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:33.702870 28977 ts_tablet_manager.cc:1905] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:33.704700 28977 ts_tablet_manager.cc:1918] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:33.704756 28977 log.cc:1196] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/114acdeeee0a4c22aeb2d4c89e267add
I20260806 03:52:33.704958 28977 ts_tablet_manager.cc:1939] T 114acdeeee0a4c22aeb2d4c89e267add P e504150cd3cb4109b93e6ca820a732ee: Deleting consensus metadata
I20260806 03:52:33.705331 28170 catalog_manager.cc:4841] TS e504150cd3cb4109b93e6ca820a732ee (127.12.15.67:38971): tablet 114acdeeee0a4c22aeb2d4c89e267add (table test-table [id=2a1968be22d3435e8cb7ff37ab429118]) successfully deleted
I20260806 03:52:33.715945 28978 raft_consensus.cc:2258] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:52:33.716354 28978 ts_tablet_manager.cc:1905] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:52:33.717649 28978 ts_tablet_manager.cc:1918] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:52:33.717717 28978 log.cc:1196] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/114acdeeee0a4c22aeb2d4c89e267add
I20260806 03:52:33.718008 28978 ts_tablet_manager.cc:1939] T 114acdeeee0a4c22aeb2d4c89e267add P 858a4bc68f2347649ba7ad09339f1385: Deleting consensus metadata
I20260806 03:52:33.718598 28171 catalog_manager.cc:4841] TS 858a4bc68f2347649ba7ad09339f1385 (127.12.15.66:42805): tablet 114acdeeee0a4c22aeb2d4c89e267add (table test-table [id=2a1968be22d3435e8cb7ff37ab429118]) successfully deleted
W20260806 03:52:33.719940 28169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:45959 (error 108)
I20260806 03:52:33.720185 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 28505
I20260806 03:52:33.720708 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:52:33.720261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:45959 (local address 127.0.0.1:52486)
0806 03:52:33.720627 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0806 03:52:33.720676 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:45959: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346}
W20260806 03:52:33.727775 28169 catalog_manager.cc:4584] TS 046cfb58f5dd4ff3a0a8f964e5192425 (127.12.15.65:45959): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 114acdeeee0a4c22aeb2d4c89e267add: Network error: Client connection negotiation failed: client connection to 127.12.15.65:45959: connect: Connection refused (error 111)
I20260806 03:52:33.731101 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 28712
I20260806 03:52:33.742235 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 28144
2026-08-06T03:52:33Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2074 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28985](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28985](info): set up 2 sockets
Aug 06 03:52:33 dist-test-slave-bqcl krb5kdc[28985](info): commencing operation
krb5kdc: starting...
W20260806 03:52:34.877475 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:52:34 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988354, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:52:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:52:34Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.15.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.15.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/rangeradmin_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/rangeradmin_127.12.15.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.15.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.15.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/rangerlookup_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/rangerlookup_127.12.15.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.15.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.15.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/HTTP_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/HTTP_127.12.15.84@KRBTEST.COM.keytab.
I20260806 03:52:35.034857 12349 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGVWHtl/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/postgres -l logfile start
2026-08-06 03:52:37.415 UTC [29028] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 03:52:37.415 UTC [29028] LOG: listening on IPv4 address "127.12.15.84", port 55439
2026-08-06 03:52:37.418 UTC [29028] LOG: listening on Unix socket "/tmp/.s.PGSQL.55439"
2026-08-06 03:52:37.438 UTC [29033] LOG: database system was shut down at 2026-08-06 03:52:36 UTC
2026-08-06 03:52:37.442 UTC [29028] LOG: database system is ready to accept connections
I20260806 03:52:38.474327 12349 mini_postgres.cc:96] Postgres bound to 55439
2026-08-06 03:52:38.482 UTC [29045] FATAL: database "slave" does not exist
127.12.15.84:55439 - accepting connections
I20260806 03:52:38.483702 12349 mini_ranger.cc:161] Starting Ranger...
I20260806 03:52:38.499388 12349 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 03:52:38.922578 12349 mini_ranger.cc:87] Created ranger Postgres database
I20260806 03:52:38.922660 12349 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-admin
2026-08-06 03:52:39,297 [I] DB FLAVOR :POSTGRES
2026-08-06 03:52:39,298 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 03:52:39,298 [I] Checking connection..
2026-08-06 03:52:39,298 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 03:52:39,585 [I] Checking connection passed.
2026-08-06 03:52:39,585 [I] --------- Verifying version history table ---------
2026-08-06 03:52:39,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:52:39,832 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 03:52:39,833 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:52:40,088 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 03:52:40,088 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 03:52:40,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 03:52:40.340 UTC [29148] WARNING: there is no transaction in progress
2026-08-06 03:52:40,348 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 03:52:40,348 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:52:40,600 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 03:52:40,600 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 03:52:40,600 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 03:52:40,844 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 03:52:41,098 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bqcl.c.gcp-upstream.internal','N') ;"
2026-08-06 03:52:41,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 03:52:41,582 [I] Table x_portal_user does not exist in database ranger
2026-08-06 03:52:41,582 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 03:52:41,812 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 03:52:41,812 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 03:52:42,069 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 03:52:42,294 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 03:52:42,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 03:52:43.199 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.214 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.230 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.245 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.261 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.278 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.342 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.349 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.363 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.372 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.382 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.396 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.405 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.411 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.417 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:43.423 UTC [29325] WARNING: there is no transaction in progress
2026-08-06 03:52:44,015 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 03:52:44,015 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 03:52:44,258 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 03:52:44,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 03:52:44,501 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 03:52:44,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 03:52:44,757 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 03:52:44,757 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 03:52:44,980 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 03:52:45,232 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bqcl.c.gcp-upstream.internal';"
2026-08-06 03:52:45,478 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 03:52:45,479 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 03:52:45,725 [I] DB_PATCHES have already been applied
I20260806 03:52:45.730501 12349 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGVWHtl/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 03:52:45.730561 12349 mini_ranger.cc:193] Using host: 127.12.15.84
I20260806 03:52:45.734184 12349 mini_ranger.cc:227] Ranger admin URL: http://127.12.15.84:35877
Aug 06, 2026 3:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 3:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 3:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 3:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 3:52:46 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 03:52:46.913444 12349 mini_ranger.cc:160] Time spent starting Ranger: real 8.430s user 0.002s sys 0.008s
Aug 06, 2026 3:53:06 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 3:53:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 3:53:06 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 3:53:06 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 3:53:07 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 03:53:08.062426 12349 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.15.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.15.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/rangerkms_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/rangerkms_127.12.15.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.15.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.15.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.15.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/HTTP_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.15.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/HTTP_127.12.15.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 03:53:08 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988388, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 03:53:08.143879 12349 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 03:53:08.153184 12349 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 03:53:08.938957 12349 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 03:53:08.939044 12349 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms
I20260806 03:53:08.939059 12349 mini_ranger_kms.cc:227] Using postgres at 127.12.15.84:55439
2026-08-06 03:53:09,570 [I] DB FLAVOR :POSTGRES
2026-08-06 03:53:09,571 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 03:53:09,571 [I] Checking connection
2026-08-06 03:53:09,571 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 03:53:09,815 [I] connection success
2026-08-06 03:53:09,815 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 03:53:09,815 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 03:53:09,815 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 03:53:10,047 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 03:53:10,048 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 03:53:10,048 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 03:53:10,048 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:55439/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 03:53:10,336 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 03:53:10.791144 12349 mini_ranger_kms.cc:310] Created kms service
I20260806 03:53:10.966224 12349 mini_ranger_kms.cc:326] Created kudu user
I20260806 03:53:11.009280 12349 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 03:53:11 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988388, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:53:11.384837 12349 mini_ranger_kms.cc:384] Added ranger policy
I20260806 03:53:11.384951 12349 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 03:53:11.384971 12349 mini_ranger_kms.cc:240] Using host: 127.12.15.84
I20260806 03:53:11.389195 12349 mini_ranger_kms.cc:276] Ranger KMS PID: 29605
I20260806 03:53:11.389328 12349 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.15.84:43481
Aug 06, 2026 3:53:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:53:12 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:53:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 3:53:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 3:53:12 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 3:53:12 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 03:53:13 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988393, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:53:13.860947 12349 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.717s user 0.000s sys 0.303s
I20260806 03:53:13.861058 12349 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 03:53:13.861132 12349 mini_ranger_kms.cc:401] 127.12.15.84:43481/kms/v1/keys
Aug 06 03:53:14 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988393, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.84@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
Aug 06 03:53:14 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988393, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.84@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
Aug 06 03:53:16 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:53:16.604657 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:44705
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:44705
--ranger_config_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:43481/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:53:16.721544 29684 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:53:16.721837 29684 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:53:16.721872 29684 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:53:16.723528 29684 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:53:16.723585 29684 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:53:16.723613 29684 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:53:16.723639 29684 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:53:16.723660 29684 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:53:16.725883 29684 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.15.84:43481/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:44705
--ranger_config_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:44705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29684
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:53:16.726147 29684 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:53:16.726411 29684 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:53:16.731460 29695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:53:16.731564 29699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:53:16.731467 29694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:53:16.731630 29684 server_base.cc:1034] running on GCE node
I20260806 03:53:16.731943 29684 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:53:16.732259 29684 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:53:16.733428 29684 hybrid_clock.cc:648] HybridClock initialized: now 1785988396733408 us; error 34 us; skew 500 ppm
Aug 06 03:53:16 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988396, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:53:16.740437 29684 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:53:16.740952 29684 webserver.cc:460] Webserver started at http://127.12.15.126:36323/ using document root <none> and password file <none>
I20260806 03:53:16.741194 29684 fs_manager.cc:359] Metadata directory not provided
I20260806 03:53:16.741245 29684 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:53:16.741374 29684 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:53:16 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988396, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:53:16.794727 29684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "159ffe667d8a4733a5d0e35a3f1a75f5"
format_stamp: "Formatted at 2026-08-06 03:53:16 on dist-test-slave-bqcl"
server_key: "d42d71a091dc628e32962f61ebad0a13"
server_key_iv: "4ddedfeb8752d32f137855d9d2150882"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:16.795262 29684 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "159ffe667d8a4733a5d0e35a3f1a75f5"
format_stamp: "Formatted at 2026-08-06 03:53:16 on dist-test-slave-bqcl"
server_key: "d42d71a091dc628e32962f61ebad0a13"
server_key_iv: "4ddedfeb8752d32f137855d9d2150882"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:16.797178 29684 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.001s
I20260806 03:53:16.811271 29708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:16.811569 29684 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:53:16.811683 29684 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "159ffe667d8a4733a5d0e35a3f1a75f5"
format_stamp: "Formatted at 2026-08-06 03:53:16 on dist-test-slave-bqcl"
server_key: "d42d71a091dc628e32962f61ebad0a13"
server_key_iv: "4ddedfeb8752d32f137855d9d2150882"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:16.811779 29684 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:53:16.849839 29684 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:53:16.852152 29684 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:53:16.852316 29684 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:53:16.857484 29684 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:44705
I20260806 03:53:16.857550 29800 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:44705 every 8 connection(s)
I20260806 03:53:16.857933 29684 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:53:16.858714 29801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:53:16.861080 29801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5: Bootstrap starting.
I20260806 03:53:16.861686 29801 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5: Neither blocks nor log segments found. Creating new log.
I20260806 03:53:16.861967 29801 log.cc:824] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5: Log is configured to *not* fsync() on all Append() calls
I20260806 03:53:16.862304 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 29684
I20260806 03:53:16.862406 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:53:16.862637 12349 external_mini_cluster.cc:1428] Setting key fe075b8abbf648a418bc054bc1872039
I20260806 03:53:16.862902 29801 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5: No bootstrap required, opened a new log
I20260806 03:53:16.865932 29801 raft_consensus.cc:348] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159ffe667d8a4733a5d0e35a3f1a75f5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44705 } }
I20260806 03:53:16.866088 29801 raft_consensus.cc:374] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:53:16.866153 29801 raft_consensus.cc:729] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 159ffe667d8a4733a5d0e35a3f1a75f5, State: Initialized, Role: FOLLOWER
I20260806 03:53:16.866331 29801 consensus_queue.cc:260] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159ffe667d8a4733a5d0e35a3f1a75f5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44705 } }
I20260806 03:53:16.866410 29801 raft_consensus.cc:388] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:53:16.866444 29801 raft_consensus.cc:482] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:53:16.866514 29801 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:53:16.867086 29801 raft_consensus.cc:504] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159ffe667d8a4733a5d0e35a3f1a75f5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44705 } }
I20260806 03:53:16.867193 29801 leader_election.cc:302] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 159ffe667d8a4733a5d0e35a3f1a75f5; no voters:
I20260806 03:53:16.867373 29801 leader_election.cc:288] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:53:16.867549 29817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:53:16.867640 29801 sys_catalog.cc:564] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:53:16.867739 29817 raft_consensus.cc:686] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [term 1 LEADER]: Becoming Leader. State: Replica: 159ffe667d8a4733a5d0e35a3f1a75f5, State: Running, Role: LEADER
I20260806 03:53:16.867913 29817 consensus_queue.cc:237] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159ffe667d8a4733a5d0e35a3f1a75f5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44705 } }
I20260806 03:53:16.869588 29813 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "159ffe667d8a4733a5d0e35a3f1a75f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159ffe667d8a4733a5d0e35a3f1a75f5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44705 } } }
I20260806 03:53:16.869635 29801 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 03:53:16.869690 29813 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [sys.catalog]: This master's current role is: LEADER
I20260806 03:53:16.869899 29818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 159ffe667d8a4733a5d0e35a3f1a75f5. Latest consensus state: current_term: 1 leader_uuid: "159ffe667d8a4733a5d0e35a3f1a75f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "159ffe667d8a4733a5d0e35a3f1a75f5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44705 } } }
I20260806 03:53:16.869961 29818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5 [sys.catalog]: This master's current role is: LEADER
I20260806 03:53:16.870342 29835 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 06 03:53:16 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:53:16.871524 29835 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:53:16.878868 29812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:16.863182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41964 (local address 127.12.15.126:44705)
0806 03:53:16.863373 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:16.863384 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:16.863443 (+ 59us) server_negotiation.cc:407] Connection header received
0806 03:53:16.864464 (+ 1021us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:16.864475 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:16.864541 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:16.864686 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:53:16.865544 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:16.867093 (+ 1549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:16.868258 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:16.868873 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:16.868981 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:16.871783 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:53:16.871788 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:53:16.871788 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:53:16.871800 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:53:16.876066 (+ 4266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:16.876611 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:16.876618 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:16.876619 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:16.876824 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:16.877066 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:16.877067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:16.877067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:16.877651 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:53:16.877918 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:16.878025 (+ 107us) server_negotiation.cc:299] Negotiation successful
0806 03:53:16.878536 (+ 511us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260806 03:53:16.883329 29835 catalog_manager.cc:1347] Generated new cluster ID: 987660c4093a46fb90f069cb4e54da82
I20260806 03:53:16.883438 29835 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:53:16.898350 29835 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:53:16.899454 29835 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:53:16.905203 29835 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5: Generated new TSK 0
I20260806 03:53:16.905398 29835 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:53:17.559396 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:43481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44705
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:53:17.669770 29878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:53:17.669960 29878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:53:17.669981 29878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:53:17.671139 29878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:53:17.671183 29878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:53:17.671217 29878 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:53:17.673031 29878 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.15.84:43481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:53:17.673282 29878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:53:17.673575 29878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:53:17.678782 29885 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:53:17.678758 29887 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:53:17.678769 29884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:53:17.679385 29878 server_base.cc:1034] running on GCE node
I20260806 03:53:17.679584 29878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:53:17.679804 29878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:53:17.680946 29878 hybrid_clock.cc:648] HybridClock initialized: now 1785988397680930 us; error 35 us; skew 500 ppm
Aug 06 03:53:17 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988397, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:53:17.684026 29878 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:53:17.684516 29878 webserver.cc:460] Webserver started at http://127.12.15.65:38597/ using document root <none> and password file <none>
I20260806 03:53:17.684752 29878 fs_manager.cc:359] Metadata directory not provided
I20260806 03:53:17.684818 29878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:53:17.684969 29878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:53:17 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988397, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:53:17.702865 29878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "abf208ff184b43978acb9ab453c68621"
format_stamp: "Formatted at 2026-08-06 03:53:17 on dist-test-slave-bqcl"
server_key: "aef1d7d19621f961d3fbff1ad82788f8"
server_key_iv: "175c089229475b237eee0ec27e79a159"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:17.703437 29878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "abf208ff184b43978acb9ab453c68621"
format_stamp: "Formatted at 2026-08-06 03:53:17 on dist-test-slave-bqcl"
server_key: "aef1d7d19621f961d3fbff1ad82788f8"
server_key_iv: "175c089229475b237eee0ec27e79a159"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:17.705281 29878 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260806 03:53:17.722286 29902 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:17.722460 29878 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.000s sys 0.001s
I20260806 03:53:17.722554 29878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "abf208ff184b43978acb9ab453c68621"
format_stamp: "Formatted at 2026-08-06 03:53:17 on dist-test-slave-bqcl"
server_key: "aef1d7d19621f961d3fbff1ad82788f8"
server_key_iv: "175c089229475b237eee0ec27e79a159"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:17.722662 29878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:53:17.739658 29878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:53:17.742033 29878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:53:17.742311 29878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:53:17.742614 29878 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:53:17.743013 29878 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:53:17.743103 29878 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:17.743165 29878 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:53:17.743208 29878 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:17.749564 29878 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:42767
I20260806 03:53:17.750015 29878 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:53:17.754341 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 29878
I20260806 03:53:17.754457 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:53:17.754676 12349 external_mini_cluster.cc:1428] Setting key 84dbfdfbbc0bd34bf9d1d530f20da2d2
I20260806 03:53:17.754961 30074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:42767 every 8 connection(s)
Aug 06 03:53:17 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988397, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:53:17.765216 30078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:17.751116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44705 (local address 127.12.15.65:49475)
0806 03:53:17.753692 (+ 2576us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:17.753729 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:53:17.754248 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:53:17.755657 (+ 1409us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:53:17.755663 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:53:17.755877 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:53:17.756155 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:17.756162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:17.757845 (+ 1683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:17.757847 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:17.758971 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:17.758974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:17.759359 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:17.759360 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:17.759500 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:17.760052 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:53:17.760072 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:53:17.761722 (+ 1650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:53:17.763544 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:53:17.763552 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:53:17.763554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:53:17.763851 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:53:17.764087 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:53:17.764088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:53:17.764088 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:53:17.764226 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:53:17.764524 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:53:17.764531 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:53:17.764697 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 03:53:17.764803 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 03:53:17.764961 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2536,"thread_start_us":80,"threads_started":1}
I20260806 03:53:17.765771 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:17.751134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:49475 (local address 127.12.15.126:44705)
0806 03:53:17.755398 (+ 4264us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:17.755401 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:17.755409 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:53:17.755431 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:17.755432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:17.755445 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:17.755506 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:53:17.757002 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:17.757742 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:17.759062 (+ 1320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:17.759282 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:17.759342 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:17.761831 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:53:17.761832 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:53:17.761833 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:53:17.761840 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:53:17.763448 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:17.763936 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:17.763938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:17.763938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:17.764004 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:17.764296 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:17.764297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:17.764297 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:17.764435 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:53:17.764495 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:17.765591 (+ 1096us) server_negotiation.cc:299] Negotiation successful
0806 03:53:17.765672 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4233,"thread_start_us":101,"threads_started":1}
I20260806 03:53:17.766618 30075 heartbeater.cc:347] Connected to a master server at 127.12.15.126:44705
I20260806 03:53:17.766710 30075 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
I20260806 03:53:17.766898 30075 heartbeater.cc:511] Master 127.12.15.126:44705 requested a full tablet report, sending...
I20260806 03:53:17.767534 29736 ts_manager.cc:194] Registered new tserver with Master: abf208ff184b43978acb9ab453c68621 (127.12.15.65:42767)
I20260806 03:53:17.768303 29736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:49475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:53:17.793358 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:43481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44705
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:53:17 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988397, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 03:53:17.915093 30084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:53:17.915438 30084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:53:17.915529 30084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:53:17.917301 30084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:53:17.917363 30084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:53:17.917403 30084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:53:17.919617 30084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.15.84:43481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:53:17.920008 30084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:53:17.920326 30084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:53:17.926877 30099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:53:17.927322 30096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:53:17.927465 30101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:53:17.928486 30084 server_base.cc:1034] running on GCE node
I20260806 03:53:17.928725 30084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:53:17.929045 30084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:53:17.930184 30084 hybrid_clock.cc:648] HybridClock initialized: now 1785988397930156 us; error 44 us; skew 500 ppm
Aug 06 03:53:17 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988397, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:53:17.933374 30084 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:53:17.933861 30084 webserver.cc:460] Webserver started at http://127.12.15.66:35911/ using document root <none> and password file <none>
I20260806 03:53:17.934101 30084 fs_manager.cc:359] Metadata directory not provided
I20260806 03:53:17.934175 30084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:53:17.934365 30084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:53:17 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988397, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:53:17.951712 30084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "f1969bae759249b290d03bb88b731901"
format_stamp: "Formatted at 2026-08-06 03:53:17 on dist-test-slave-bqcl"
server_key: "f0c8a275a14b0a05fe71a78b1263d3be"
server_key_iv: "f79f42175291cd925458f22bb2f39c42"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:17.952173 30084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "f1969bae759249b290d03bb88b731901"
format_stamp: "Formatted at 2026-08-06 03:53:17 on dist-test-slave-bqcl"
server_key: "f0c8a275a14b0a05fe71a78b1263d3be"
server_key_iv: "f79f42175291cd925458f22bb2f39c42"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:17.958619 30084 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.000s
I20260806 03:53:17.971499 30113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:17.971735 30084 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:53:17.971836 30084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "f1969bae759249b290d03bb88b731901"
format_stamp: "Formatted at 2026-08-06 03:53:17 on dist-test-slave-bqcl"
server_key: "f0c8a275a14b0a05fe71a78b1263d3be"
server_key_iv: "f79f42175291cd925458f22bb2f39c42"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:17.971926 30084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:53:17.987094 30084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:53:17.989677 30084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:53:17.989884 30084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:53:17.990247 30084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:53:17.990684 30084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:53:17.990777 30084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:17.990851 30084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:53:17.990911 30084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:17.997550 30084 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:43415
I20260806 03:53:17.998066 30084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:53:17.998299 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 30084
I20260806 03:53:17.998430 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:53:17.998661 12349 external_mini_cluster.cc:1428] Setting key dae2885f8b61202fd45b8da13849f994
Aug 06 03:53:18 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988397, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:53:18.010608 30310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:43415 every 8 connection(s)
I20260806 03:53:18.011610 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.000324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:48349 (local address 127.12.15.126:44705)
0806 03:53:18.000368 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.000371 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.000577 (+ 206us) server_negotiation.cc:407] Connection header received
0806 03:53:18.001888 (+ 1311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.001890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.001907 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.001938 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:53:18.002814 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.003516 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.005025 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.005334 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.005383 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.008061 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:53:18.008062 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:53:18.008063 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:53:18.008070 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:53:18.009714 (+ 1644us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:18.010359 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:18.010360 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:18.010361 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:18.010433 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:18.010822 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:18.010823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:18.010823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:18.010971 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:53:18.011041 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.011417 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.011514 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:53:18.012269 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.000041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44705 (local address 127.12.15.66:48349)
0806 03:53:18.000447 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.000494 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0806 03:53:18.001763 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:53:18.002159 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:53:18.002163 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:53:18.002410 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:53:18.002704 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.002710 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.003635 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.003636 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.004903 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.004906 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.005479 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.005480 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.005624 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.006268 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:53:18.006278 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:53:18.007934 (+ 1656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:53:18.009836 (+ 1902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:53:18.009842 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:53:18.009845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:53:18.010215 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:53:18.010507 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:53:18.010508 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:53:18.010508 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:53:18.010652 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:53:18.011078 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:53:18.011085 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:53:18.011249 (+ 164us) client_negotiation.cc:769] Sending connection context
0806 03:53:18.011849 (+ 600us) client_negotiation.cc:240] Negotiation successful
0806 03:53:18.012017 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":84,"threads_started":1}
I20260806 03:53:18.012914 30313 heartbeater.cc:347] Connected to a master server at 127.12.15.126:44705
I20260806 03:53:18.013012 30313 heartbeater.cc:464] Registering TS with master...
I20260806 03:53:18.013224 30313 heartbeater.cc:511] Master 127.12.15.126:44705 requested a full tablet report, sending...
I20260806 03:53:18.013634 29736 ts_manager.cc:194] Registered new tserver with Master: f1969bae759249b290d03bb88b731901 (127.12.15.66:43415)
I20260806 03:53:18.014113 29736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:48349
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:53:18.042899 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:43481/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44705
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:33971
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:53:18.165321 30318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:53:18.165619 30318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:53:18.165655 30318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:53:18.167343 30318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:53:18.167397 30318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:53:18.167433 30318 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:53:18.169700 30318 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33971
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.15.84:43481/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:44705
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:53:18.169958 30318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:53:18.170296 30318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:53:18.175822 30326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:53:18.175830 30331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:53:18.177204 30327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:53:18.177847 30318 server_base.cc:1034] running on GCE node
I20260806 03:53:18.178073 30318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:53:18.178370 30318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:53:18.179522 30318 hybrid_clock.cc:648] HybridClock initialized: now 1785988398179502 us; error 36 us; skew 500 ppm
Aug 06 03:53:18 dist-test-slave-bqcl krb5kdc[28985](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988398, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:53:18.184062 30318 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:53:18.184624 30318 webserver.cc:460] Webserver started at http://127.12.15.67:39467/ using document root <none> and password file <none>
I20260806 03:53:18.184883 30318 fs_manager.cc:359] Metadata directory not provided
I20260806 03:53:18.184931 30318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:53:18.185063 30318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:53:18 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988398, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:53:18.203497 30318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "f988a975db68436cb695bac1fbcf8c41"
format_stamp: "Formatted at 2026-08-06 03:53:18 on dist-test-slave-bqcl"
server_key: "3a954d4252afd22a7686ccebc82fd5da"
server_key_iv: "4891153c40655250acef8ac19ee92991"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:18.203975 30318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "f988a975db68436cb695bac1fbcf8c41"
format_stamp: "Formatted at 2026-08-06 03:53:18 on dist-test-slave-bqcl"
server_key: "3a954d4252afd22a7686ccebc82fd5da"
server_key_iv: "4891153c40655250acef8ac19ee92991"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:18.205961 30318 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260806 03:53:18.222260 30340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:18.222479 30318 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260806 03:53:18.222574 30318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "f988a975db68436cb695bac1fbcf8c41"
format_stamp: "Formatted at 2026-08-06 03:53:18 on dist-test-slave-bqcl"
server_key: "3a954d4252afd22a7686ccebc82fd5da"
server_key_iv: "4891153c40655250acef8ac19ee92991"
server_key_version: "kuduclusterkey@0"
I20260806 03:53:18.222668 30318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:53:18.235898 30318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:53:18.238740 30318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:53:18.238936 30318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:53:18.239187 30318 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:53:18.239578 30318 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:53:18.239632 30318 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:18.239663 30318 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:53:18.239687 30318 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:53:18.251545 30318 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:38497
I20260806 03:53:18.252007 30318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:53:18.253924 30522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:38497 every 8 connection(s)
Aug 06 03:53:18 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988398, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:53:18.261021 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 30318
I20260806 03:53:18.261125 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:53:18.261303 12349 external_mini_cluster.cc:1428] Setting key 10bf67687885f8005cace6c1e205fff0
I20260806 03:53:18.264660 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.253186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:46405 (local address 127.12.15.126:44705)
0806 03:53:18.253228 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.253230 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.253241 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:53:18.253973 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.253975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.253989 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.254026 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:53:18.254998 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.255801 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.257341 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.257679 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.257724 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.260425 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:53:18.260426 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:53:18.260426 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:53:18.260433 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:53:18.263009 (+ 2576us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:18.263496 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:18.263497 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:18.263498 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:18.263577 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:18.263909 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:18.263910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:18.263910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:18.264048 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:53:18.264092 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.264464 (+ 372us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.264566 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:53:18.265431 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.252783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44705 (local address 127.12.15.67:46405)
0806 03:53:18.253094 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.253144 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 03:53:18.253823 (+ 679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:53:18.254318 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:53:18.254322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:53:18.254594 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:53:18.254879 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.254886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.255924 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.255926 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.257230 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.257233 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.257820 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.257821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.257980 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.258590 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:53:18.258600 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:53:18.260319 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:53:18.263111 (+ 2792us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:53:18.263119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:53:18.263121 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:53:18.263416 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:53:18.263652 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:53:18.263653 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:53:18.263653 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:53:18.263832 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:53:18.264139 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:53:18.264145 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:53:18.264307 (+ 162us) client_negotiation.cc:769] Sending connection context
0806 03:53:18.265009 (+ 702us) client_negotiation.cc:240] Negotiation successful
0806 03:53:18.265179 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":195,"threads_started":1}
I20260806 03:53:18.271816 30524 heartbeater.cc:347] Connected to a master server at 127.12.15.126:44705
I20260806 03:53:18.271921 30524 heartbeater.cc:464] Registering TS with master...
I20260806 03:53:18.272121 30524 heartbeater.cc:511] Master 127.12.15.126:44705 requested a full tablet report, sending...
I20260806 03:53:18.272480 29736 ts_manager.cc:194] Registered new tserver with Master: f988a975db68436cb695bac1fbcf8c41 (127.12.15.67:38497)
I20260806 03:53:18.272995 29736 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:46405
I20260806 03:53:18.274541 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:53:18.290164 30079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.277010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41972 (local address 127.12.15.126:44705)
0806 03:53:18.277049 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.277051 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.277327 (+ 276us) server_negotiation.cc:407] Connection header received
0806 03:53:18.277890 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.277892 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.277905 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.277944 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:53:18.283462 (+ 5518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.284191 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.285345 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.285662 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.285704 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.287286 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:53:18.287287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:53:18.287288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:53:18.287294 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:53:18.288911 (+ 1617us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:18.289324 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:18.289325 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:18.289326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:18.289383 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:53:18.289565 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:53:18.289566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:53:18.289566 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:53:18.289684 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:53:18.289753 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.289955 (+ 202us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.290047 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:53:18.292089 29736 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41972:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:53:18.294368 29736 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:53:18.313593 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.297686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38497 (local address 127.0.0.1:52280)
0806 03:53:18.299531 (+ 1845us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.299573 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:53:18.299639 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:53:18.300723 (+ 1084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:53:18.300724 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:53:18.300726 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:53:18.300756 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.300757 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.306092 (+ 5335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.306093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.307655 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.307659 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.313275 (+ 5616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.313276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.313363 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.313378 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:53:18.313450 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:53:18.313504 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1812,"thread_start_us":74,"threads_started":1}
I20260806 03:53:18.314491 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.297985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:43415 (local address 127.0.0.1:53218)
0806 03:53:18.300136 (+ 2151us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.300146 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:53:18.300201 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:53:18.300572 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:53:18.300575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:53:18.300578 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:53:18.300648 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.300650 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.303008 (+ 2358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.303009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.310201 (+ 7192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.310204 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.314250 (+ 4046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.314250 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.314316 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.314320 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:53:18.314373 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:53:18.314412 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2135,"thread_start_us":90,"threads_started":1}
I20260806 03:53:18.314767 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.298889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:42767 (local address 127.0.0.1:46882)
0806 03:53:18.302688 (+ 3799us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.302699 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:53:18.302749 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:53:18.303716 (+ 967us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:53:18.303717 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:53:18.303719 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:53:18.303759 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.303760 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.307759 (+ 3999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.307760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.308857 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.308860 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.314549 (+ 5689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.314550 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.314617 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.314620 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:53:18.314668 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 03:53:18.314701 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3781,"thread_start_us":73,"threads_started":1}
I20260806 03:53:18.315685 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.298366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52280 (local address 127.12.15.67:38497)
0806 03:53:18.299784 (+ 1418us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.299789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.299799 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:53:18.299841 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.299843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.299862 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.299936 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:53:18.300826 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.305974 (+ 5148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.312326 (+ 6352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.313089 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.313163 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.313235 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.315561 (+ 2326us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.315604 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1381,"thread_start_us":88,"threads_started":1}
I20260806 03:53:18.316179 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.298587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53218 (local address 127.12.15.66:43415)
0806 03:53:18.300080 (+ 1493us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.300084 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.300336 (+ 252us) server_negotiation.cc:407] Connection header received
0806 03:53:18.300380 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.300382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.300397 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.300475 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:53:18.301422 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.302560 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.311304 (+ 8744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.312111 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.312187 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.312265 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.316045 (+ 3780us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.316092 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1456,"thread_start_us":98,"threads_started":1}
I20260806 03:53:18.317219 30419 tablet_service.cc:1467] Processing CreateTablet for tablet 52fc38c0724c4ffa80955bbc1246fe81 (DEFAULT_TABLE table=test-table [id=e4b2fbeb157f4a30921bd9f5eb5574e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:53:18.317620 30419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52fc38c0724c4ffa80955bbc1246fe81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:53:18.316434 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.299143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46882 (local address 127.12.15.65:42767)
0806 03:53:18.300308 (+ 1165us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.300313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.302809 (+ 2496us) server_negotiation.cc:407] Connection header received
0806 03:53:18.302849 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.302851 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.302868 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.302935 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:53:18.303845 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.304959 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.310299 (+ 5340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.311093 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.311170 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.311238 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.315478 (+ 4240us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.315910 (+ 432us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1131,"thread_start_us":66,"threads_started":1}
I20260806 03:53:18.323107 29973 tablet_service.cc:1467] Processing CreateTablet for tablet 52fc38c0724c4ffa80955bbc1246fe81 (DEFAULT_TABLE table=test-table [id=e4b2fbeb157f4a30921bd9f5eb5574e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:53:18.323489 29973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52fc38c0724c4ffa80955bbc1246fe81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:53:18.327601 30558 tablet_bootstrap.cc:492] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: Bootstrap starting.
I20260806 03:53:18.327915 30197 tablet_service.cc:1467] Processing CreateTablet for tablet 52fc38c0724c4ffa80955bbc1246fe81 (DEFAULT_TABLE table=test-table [id=e4b2fbeb157f4a30921bd9f5eb5574e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:53:18.328292 30197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 52fc38c0724c4ffa80955bbc1246fe81. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:53:18.329344 30558 tablet_bootstrap.cc:654] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: Neither blocks nor log segments found. Creating new log.
I20260806 03:53:18.329723 30558 log.cc:824] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: Log is configured to *not* fsync() on all Append() calls
I20260806 03:53:18.329933 30559 tablet_bootstrap.cc:492] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621: Bootstrap starting.
I20260806 03:53:18.330682 30559 tablet_bootstrap.cc:654] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621: Neither blocks nor log segments found. Creating new log.
I20260806 03:53:18.331321 30559 log.cc:824] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621: Log is configured to *not* fsync() on all Append() calls
I20260806 03:53:18.332171 30558 tablet_bootstrap.cc:492] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: No bootstrap required, opened a new log
I20260806 03:53:18.332291 30558 ts_tablet_manager.cc:1397] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260806 03:53:18.334373 30558 raft_consensus.cc:348] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.334636 30558 raft_consensus.cc:374] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:53:18.334721 30558 raft_consensus.cc:729] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f988a975db68436cb695bac1fbcf8c41, State: Initialized, Role: FOLLOWER
I20260806 03:53:18.334910 30558 consensus_queue.cc:260] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.335263 30558 ts_tablet_manager.cc:1428] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:53:18.335417 30524 heartbeater.cc:503] Master 127.12.15.126:44705 was elected leader, sending a full tablet report...
I20260806 03:53:18.337113 30559 tablet_bootstrap.cc:492] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621: No bootstrap required, opened a new log
I20260806 03:53:18.337231 30559 ts_tablet_manager.cc:1397] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260806 03:53:18.339716 30559 raft_consensus.cc:348] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.339931 30559 raft_consensus.cc:374] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:53:18.340005 30559 raft_consensus.cc:729] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abf208ff184b43978acb9ab453c68621, State: Initialized, Role: FOLLOWER
I20260806 03:53:18.340209 30559 consensus_queue.cc:260] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.340512 30559 ts_tablet_manager.cc:1428] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:53:18.340658 30075 heartbeater.cc:503] Master 127.12.15.126:44705 was elected leader, sending a full tablet report...
I20260806 03:53:18.342274 30563 tablet_bootstrap.cc:492] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: Bootstrap starting.
I20260806 03:53:18.343314 30563 tablet_bootstrap.cc:654] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: Neither blocks nor log segments found. Creating new log.
I20260806 03:53:18.343675 30563 log.cc:824] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: Log is configured to *not* fsync() on all Append() calls
I20260806 03:53:18.344416 30563 tablet_bootstrap.cc:492] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: No bootstrap required, opened a new log
I20260806 03:53:18.344480 30563 ts_tablet_manager.cc:1397] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:53:18.346801 30563 raft_consensus.cc:348] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.346957 30563 raft_consensus.cc:374] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:53:18.346987 30563 raft_consensus.cc:729] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1969bae759249b290d03bb88b731901, State: Initialized, Role: FOLLOWER
I20260806 03:53:18.347098 30563 consensus_queue.cc:260] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.347334 30563 ts_tablet_manager.cc:1428] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:53:18.347466 30313 heartbeater.cc:503] Master 127.12.15.126:44705 was elected leader, sending a full tablet report...
I20260806 03:53:18.395252 30567 raft_consensus.cc:482] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:53:18.395397 30567 raft_consensus.cc:504] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.395771 30567 leader_election.cc:288] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f988a975db68436cb695bac1fbcf8c41 (127.12.15.67:38497), f1969bae759249b290d03bb88b731901 (127.12.15.66:43415)
I20260806 03:53:18.415153 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.396594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:36129 (local address 127.12.15.67:38497)
0806 03:53:18.396684 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.396687 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.406326 (+ 9639us) server_negotiation.cc:407] Connection header received
0806 03:53:18.406436 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.406438 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.406449 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.406476 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:53:18.406722 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.407438 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.414218 (+ 6780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.414768 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.414815 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.414824 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.415033 (+ 209us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.415073 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 03:53:18.415419 30572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.396009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38497 (local address 127.12.15.65:36129)
0806 03:53:18.406232 (+ 10223us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.406248 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0806 03:53:18.406362 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:53:18.406555 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:53:18.406555 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:53:18.406569 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:53:18.406653 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.406655 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.407570 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.407571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.408783 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.408787 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.414885 (+ 6098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.414886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.414962 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.414965 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:53:18.415284 (+ 319us) client_negotiation.cc:240] Negotiation successful
0806 03:53:18.415333 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10196,"thread_start_us":120,"threads_started":1}
I20260806 03:53:18.415771 30460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52fc38c0724c4ffa80955bbc1246fe81" candidate_uuid: "abf208ff184b43978acb9ab453c68621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f988a975db68436cb695bac1fbcf8c41" is_pre_election: true
I20260806 03:53:18.415977 30460 raft_consensus.cc:2454] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abf208ff184b43978acb9ab453c68621 in term 0.
I20260806 03:53:18.416266 29904 leader_election.cc:302] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abf208ff184b43978acb9ab453c68621, f988a975db68436cb695bac1fbcf8c41; no voters:
I20260806 03:53:18.416425 30567 raft_consensus.cc:2781] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:53:18.416481 30567 raft_consensus.cc:482] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:53:18.416507 30567 raft_consensus.cc:3037] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:53:18.417428 30567 raft_consensus.cc:504] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.417590 30567 leader_election.cc:288] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [CANDIDATE]: Term 1 election: Requested vote from peers f988a975db68436cb695bac1fbcf8c41 (127.12.15.67:38497), f1969bae759249b290d03bb88b731901 (127.12.15.66:43415)
I20260806 03:53:18.417793 30460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52fc38c0724c4ffa80955bbc1246fe81" candidate_uuid: "abf208ff184b43978acb9ab453c68621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f988a975db68436cb695bac1fbcf8c41"
I20260806 03:53:18.417881 30460 raft_consensus.cc:3037] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:53:18.418812 30460 raft_consensus.cc:2454] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abf208ff184b43978acb9ab453c68621 in term 1.
I20260806 03:53:18.418985 29904 leader_election.cc:302] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: abf208ff184b43978acb9ab453c68621, f988a975db68436cb695bac1fbcf8c41; no voters:
I20260806 03:53:18.419070 30567 raft_consensus.cc:2781] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:53:18.419234 30567 raft_consensus.cc:686] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [term 1 LEADER]: Becoming Leader. State: Replica: abf208ff184b43978acb9ab453c68621, State: Running, Role: LEADER
I20260806 03:53:18.419344 30551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.396641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:49465 (local address 127.12.15.66:43415)
0806 03:53:18.396717 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.396719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.415733 (+ 19014us) server_negotiation.cc:407] Connection header received
0806 03:53:18.415828 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.415829 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.415839 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.415865 (+ 26us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:53:18.416076 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.416768 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.418467 (+ 1699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.418983 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.419026 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.419034 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.419239 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.419276 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260806 03:53:18.419363 30567 consensus_queue.cc:237] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } }
I20260806 03:53:18.419600 30574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.396248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:43415 (local address 127.12.15.65:49465)
0806 03:53:18.415669 (+ 19421us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.415679 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:53:18.415761 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:53:18.415925 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:53:18.415927 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:53:18.415928 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:53:18.415988 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.415990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.416887 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.416888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.418364 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:53:18.418368 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.419092 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.419093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:53:18.419166 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.419170 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:53:18.419462 (+ 292us) client_negotiation.cc:240] Negotiation successful
0806 03:53:18.419502 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19402,"thread_start_us":575,"threads_started":1}
I20260806 03:53:18.420084 30225 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52fc38c0724c4ffa80955bbc1246fe81" candidate_uuid: "abf208ff184b43978acb9ab453c68621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1969bae759249b290d03bb88b731901" is_pre_election: true
I20260806 03:53:18.420238 30225 raft_consensus.cc:2454] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate abf208ff184b43978acb9ab453c68621 in term 0.
I20260806 03:53:18.420203 29736 catalog_manager.cc:5495] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 reported cstate change: term changed from 0 to 1, leader changed from <none> to abf208ff184b43978acb9ab453c68621 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "abf208ff184b43978acb9ab453c68621" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "abf208ff184b43978acb9ab453c68621" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 42767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 } health_report { overall_health: UNKNOWN } } }
I20260806 03:53:18.420364 30224 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "52fc38c0724c4ffa80955bbc1246fe81" candidate_uuid: "abf208ff184b43978acb9ab453c68621" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f1969bae759249b290d03bb88b731901"
I20260806 03:53:18.420437 30224 raft_consensus.cc:3037] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:53:18.421280 30224 raft_consensus.cc:2454] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate abf208ff184b43978acb9ab453c68621 in term 1.
Aug 06 03:53:18 dist-test-slave-bqcl krb5kdc[28985](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988397, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:53:18.451710 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.448300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46894 (local address 127.12.15.65:42767)
0806 03:53:18.448350 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:53:18.448352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:53:18.448363 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:53:18.448406 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:53:18.448406 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:53:18.448428 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:53:18.448505 (+ 77us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:53:18.448708 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.449396 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.450580 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:53:18.450880 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:53:18.450942 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:53:18.451146 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:53:18.451279 (+ 133us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:53:18.451348 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:53:18.451583 (+ 235us) server_negotiation.cc:299] Negotiation successful
0806 03:53:18.451607 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:53:18.456629 30224 raft_consensus.cc:1261] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 1 FOLLOWER]: Refusing update from remote peer abf208ff184b43978acb9ab453c68621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:53:18.456687 30460 raft_consensus.cc:1261] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 1 FOLLOWER]: Refusing update from remote peer abf208ff184b43978acb9ab453c68621: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:53:18.457024 30567 consensus_queue.cc:1035] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f988a975db68436cb695bac1fbcf8c41" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38497 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:53:18.457166 30575 consensus_queue.cc:1035] T 52fc38c0724c4ffa80955bbc1246fe81 P abf208ff184b43978acb9ab453c68621 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f1969bae759249b290d03bb88b731901" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 43415 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:53:18.463822 30579 mvcc.cc:205] Tried to move back new op lower bound from 7315408480075554816 to 7315408479926358016. Current Snapshot: MvccSnapshot[applied={T|T < 7315408480075554816 or (T in {7315408480075554816})}]
I20260806 03:53:18.464520 29736 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:53:18.464609 29736 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41972:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:53:18.465363 29736 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 159ffe667d8a4733a5d0e35a3f1a75f5: Sending DeleteTablet for 3 replicas of tablet 52fc38c0724c4ffa80955bbc1246fe81
I20260806 03:53:18.465694 30197 tablet_service.cc:1514] Processing DeleteTablet for tablet 52fc38c0724c4ffa80955bbc1246fe81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:53:18 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:53218
I20260806 03:53:18.466317 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 29878
I20260806 03:53:18.471850 30582 mvcc.cc:205] Tried to move back new op lower bound from 7315408480075554816 to 7315408479926358016. Current Snapshot: MvccSnapshot[applied={T|T < 7315408480075554816}]
I20260806 03:53:18.471971 30581 mvcc.cc:205] Tried to move back new op lower bound from 7315408480075554816 to 7315408479926358016. Current Snapshot: MvccSnapshot[applied={T|T < 7315408480075554816}]
I20260806 03:53:18.479375 30419 tablet_service.cc:1514] Processing DeleteTablet for tablet 52fc38c0724c4ffa80955bbc1246fe81 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:53:18 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:52280
I20260806 03:53:18.484289 30594 tablet_replica.cc:330] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: stopping tablet replica
I20260806 03:53:18.484504 30594 raft_consensus.cc:2229] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:53:18.484647 30594 raft_consensus.cc:2258] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:53:18.486550 30591 tablet_replica.cc:330] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: stopping tablet replica
I20260806 03:53:18.487012 30591 raft_consensus.cc:2229] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:53:18.487360 30591 raft_consensus.cc:2258] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:53:18.487990 30591 ts_tablet_manager.cc:1905] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:53:18.489701 30591 ts_tablet_manager.cc:1918] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:53:18.489781 30591 log.cc:1196] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/52fc38c0724c4ffa80955bbc1246fe81
I20260806 03:53:18.490044 30591 ts_tablet_manager.cc:1939] T 52fc38c0724c4ffa80955bbc1246fe81 P f1969bae759249b290d03bb88b731901: Deleting consensus metadata
I20260806 03:53:18.490530 29711 catalog_manager.cc:4841] TS f1969bae759249b290d03bb88b731901 (127.12.15.66:43415): tablet 52fc38c0724c4ffa80955bbc1246fe81 (table test-table [id=e4b2fbeb157f4a30921bd9f5eb5574e5]) successfully deleted
I20260806 03:53:18.494171 30594 ts_tablet_manager.cc:1905] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:53:18.495664 30594 ts_tablet_manager.cc:1918] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:53:18.495730 30594 log.cc:1196] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/52fc38c0724c4ffa80955bbc1246fe81
I20260806 03:53:18.496287 30594 ts_tablet_manager.cc:1939] T 52fc38c0724c4ffa80955bbc1246fe81 P f988a975db68436cb695bac1fbcf8c41: Deleting consensus metadata
I20260806 03:53:18.496944 29712 catalog_manager.cc:4841] TS f988a975db68436cb695bac1fbcf8c41 (127.12.15.67:38497): tablet 52fc38c0724c4ffa80955bbc1246fe81 (table test-table [id=e4b2fbeb157f4a30921bd9f5eb5574e5]) successfully deleted
W20260806 03:53:18.500131 29714 connection.cc:535] client connection to 127.12.15.65:42767 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260806 03:53:18.500242 29714 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:53:18.503578 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 30084
I20260806 03:53:18.526553 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.526394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:42767 (local address 127.0.0.1:46896)
0806 03:53:18.526449 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.526505 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:53:18.526868 29714 catalog_manager.cc:4584] TS abf208ff184b43978acb9ab453c68621 (127.12.15.65:42767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 52fc38c0724c4ffa80955bbc1246fe81: Network error: Client connection negotiation failed: client connection to 127.12.15.65:42767: connect: Connection refused (error 111)
I20260806 03:53:18.571586 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 30318
I20260806 03:53:18.580180 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.580022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:42767 (local address 127.0.0.1:46910)
0806 03:53:18.580091 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.580130 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260806 03:53:18.580704 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:53:18.580622 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:42767 (local address 127.0.0.1:46912)
0806 03:53:18.580660 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:53:18.580681 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:42767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
I20260806 03:53:18.592887 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 29684
I20260806 03:53:18.607967 12349 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 03:53:18.951643 12349 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 03:53:18.951745 12349 mini_ranger.cc:66] Stopping Ranger...
I20260806 03:53:19.295259 12349 mini_ranger.cc:68] Stopped Ranger
2026-08-06 03:53:19.295 UTC [29028] LOG: received smart shutdown request
2026-08-06 03:53:19.299 UTC [29028] LOG: background worker "logical replication launcher" (PID 29042) exited with exit code 1
2026-08-06 03:53:19.299 UTC [29035] LOG: shutting down
2026-08-06 03:53:19.441 UTC [29028] LOG: database system is shut down
2026-08-06T03:53:19Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (45789 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:53:19 dist-test-slave-bqcl krb5kdc[30607](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:53:19 dist-test-slave-bqcl krb5kdc[30607](info): set up 2 sockets
Aug 06 03:53:19 dist-test-slave-bqcl krb5kdc[30607](info): commencing operation
krb5kdc: starting...
W20260806 03:53:20.668264 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:53:20 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:53:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:53:20Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.12.15.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.12.15.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/rangeradmin_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/rangeradmin_127.12.15.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.12.15.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.12.15.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/rangerlookup_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/rangerlookup_127.12.15.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.15.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.12.15.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/HTTP_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/HTTP_127.12.15.84@KRBTEST.COM.keytab.
I20260806 03:53:20.829074 12349 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskGVWHtl/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/postgres -l logfile start
2026-08-06 03:53:23.119 UTC [30653] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-06 03:53:23.119 UTC [30653] LOG: listening on IPv4 address "127.12.15.84", port 53567
2026-08-06 03:53:23.122 UTC [30653] LOG: listening on Unix socket "/tmp/.s.PGSQL.53567"
2026-08-06 03:53:23.142 UTC [30658] LOG: database system was shut down at 2026-08-06 03:53:21 UTC
2026-08-06 03:53:23.146 UTC [30653] LOG: database system is ready to accept connections
I20260806 03:53:24.184265 12349 mini_postgres.cc:96] Postgres bound to 53567
2026-08-06 03:53:24.191 UTC [30670] FATAL: database "slave" does not exist
127.12.15.84:53567 - accepting connections
I20260806 03:53:24.191854 12349 mini_ranger.cc:161] Starting Ranger...
I20260806 03:53:24.206004 12349 mini_ranger.cc:84] Created miniranger Postgres user
I20260806 03:53:24.622975 12349 mini_ranger.cc:87] Created ranger Postgres database
I20260806 03:53:24.623059 12349 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-admin
2026-08-06 03:53:24,894 [I] DB FLAVOR :POSTGRES
2026-08-06 03:53:24,895 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 03:53:24,895 [I] Checking connection..
2026-08-06 03:53:24,895 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-06 03:53:25,149 [I] Checking connection passed.
2026-08-06 03:53:25,150 [I] --------- Verifying version history table ---------
2026-08-06 03:53:25,150 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:53:25,392 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 03:53:25,392 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:53:25,623 [I] Table x_db_version_h does not exist in database ranger
2026-08-06 03:53:25,624 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-06 03:53:25,624 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-06 03:53:25.877 UTC [30780] WARNING: there is no transaction in progress
2026-08-06 03:53:25,885 [I] create_dbversion_catalog.sql file imported successfully
2026-08-06 03:53:25,885 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-06 03:53:26,127 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-06 03:53:26,127 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 03:53:26,127 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-06 03:53:26,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-06 03:53:26,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-bqcl.c.gcp-upstream.internal','N') ;"
2026-08-06 03:53:26,839 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 03:53:27,090 [I] Table x_portal_user does not exist in database ranger
2026-08-06 03:53:27,090 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 03:53:27,359 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-06 03:53:27,359 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 03:53:27,609 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 03:53:27,871 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-06 03:53:27,871 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-06 03:53:28.773 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.788 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.802 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.816 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.830 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.846 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.896 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.902 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.914 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.923 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.933 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.945 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.954 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.960 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.966 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:28.972 UTC [30958] WARNING: there is no transaction in progress
2026-08-06 03:53:29,561 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-06 03:53:29,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-06 03:53:29,807 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-06 03:53:29,808 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-06 03:53:30,050 [I] Table x_portal_user already exists in database 'ranger'
2026-08-06 03:53:30,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-06 03:53:30,290 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-06 03:53:30,291 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-06 03:53:30,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-06 03:53:30,778 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-bqcl.c.gcp-upstream.internal';"
2026-08-06 03:53:31,006 [I] CORE_DB_SCHEMA import status has been updated
2026-08-06 03:53:31,007 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-06 03:53:31,245 [I] DB_PATCHES have already been applied
I20260806 03:53:31.250851 12349 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-admin:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskGVWHtl/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260806 03:53:31.250928 12349 mini_ranger.cc:193] Using host: 127.12.15.84
I20260806 03:53:31.254889 12349 mini_ranger.cc:227] Ranger admin URL: http://127.12.15.84:38617
Aug 06, 2026 3:53:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:53:31 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:53:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-admin/ews/webapp
Aug 06, 2026 3:53:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 06, 2026 3:53:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 06, 2026 3:53:32 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-admin/ews/webapp].
Aug 06, 2026 3:53:32 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260806 03:53:32.475046 12349 mini_ranger.cc:160] Time spent starting Ranger: real 8.283s user 0.001s sys 0.008s
Aug 06, 2026 3:53:52 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 06, 2026 3:53:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 06, 2026 3:53:52 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 06, 2026 3:53:52 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 06, 2026 3:53:53 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260806 03:53:54.074503 12349 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.12.15.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.12.15.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/rangerkms_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.12.15.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/rangerkms_127.12.15.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.12.15.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.12.15.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.12.15.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/HTTP_127.12.15.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.12.15.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/HTTP_127.12.15.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 06 03:53:54 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988434, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260806 03:53:54.157635 12349 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260806 03:53:54.167493 12349 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260806 03:53:54.942840 12349 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260806 03:53:54.942915 12349 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms
I20260806 03:53:54.942934 12349 mini_ranger_kms.cc:227] Using postgres at 127.12.15.84:53567
2026-08-06 03:53:55,113 [I] DB FLAVOR :POSTGRES
2026-08-06 03:53:55,113 [I] --------- Verifying Ranger DB connection ---------
2026-08-06 03:53:55,113 [I] Checking connection
2026-08-06 03:53:55,113 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-06 03:53:55,373 [I] connection success
2026-08-06 03:53:55,373 [I] --------- Verifying Ranger DB tables ---------
2026-08-06 03:53:55,373 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-06 03:53:55,373 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-06 03:53:55,608 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-06 03:53:55,608 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-06 03:53:55,608 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-06 03:53:55,608 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.12.15.84:53567/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-06 03:53:55,853 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260806 03:53:56.315719 12349 mini_ranger_kms.cc:310] Created kms service
I20260806 03:53:56.522349 12349 mini_ranger_kms.cc:326] Created kudu user
I20260806 03:53:56.561412 12349 mini_ranger_kms.cc:343] Created rangerkms user
Aug 06 03:53:56 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988434, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:53:56.939240 12349 mini_ranger_kms.cc:384] Added ranger policy
I20260806 03:53:56.939361 12349 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms:/tmp/dist-test-taskGVWHtl/build/release/bin/postgresql.jar:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskGVWHtl/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260806 03:53:56.939379 12349 mini_ranger_kms.cc:240] Using host: 127.12.15.84
I20260806 03:53:56.943557 12349 mini_ranger_kms.cc:276] Ranger KMS PID: 31239
I20260806 03:53:56.943634 12349 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.12.15.84:56591
Aug 06, 2026 3:53:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 06, 2026 3:53:57 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 06, 2026 3:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 06, 2026 3:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 06, 2026 3:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-kms/ews/webapp].
Aug 06, 2026 3:53:58 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 06 03:53:59 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988439, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:53:59.421566 12349 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.264s user 0.009s sys 0.145s
I20260806 03:53:59.421679 12349 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260806 03:53:59.421727 12349 mini_ranger_kms.cc:401] 127.12.15.84:56591/kms/v1/keys
Aug 06 03:53:59 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988439, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.84@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
Aug 06 03:54:00 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988439, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.12.15.84@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
Aug 06 03:54:02 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:54:02.298444 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37275
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:37275
--ranger_config_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:56591/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:02.380221 31322 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:02.380434 31322 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:02.380460 31322 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:02.381605 31322 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:02.381649 31322 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:02.381661 31322 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:02.381673 31322 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:02.381696 31322 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:02.383270 31322 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
--ranger_kms_url=127.12.15.84:56591/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:37275
--ranger_config_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31322
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:02.383497 31322 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:02.383703 31322 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:02.388835 31330 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:02.388862 31329 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:02.388892 31322 server_base.cc:1034] running on GCE node
W20260806 03:54:02.388873 31333 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:02.389272 31322 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:02.389545 31322 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:02.390681 31322 hybrid_clock.cc:648] HybridClock initialized: now 1785988442390664 us; error 35 us; skew 500 ppm
Aug 06 03:54:02 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988442, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:02.393103 31322 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:02.393589 31322 webserver.cc:460] Webserver started at http://127.12.15.126:33205/ using document root <none> and password file <none>
I20260806 03:54:02.393786 31322 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:02.393826 31322 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:02.393956 31322 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:54:02 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988442, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:02.441581 31322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "09198cb684a74ac7b67d55eed7261f41"
format_stamp: "Formatted at 2026-08-06 03:54:02 on dist-test-slave-bqcl"
server_key: "939752c11d58bf1396ee46c90a20931b"
server_key_iv: "342230c665987b19da85d94869a2fe3f"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:02.441998 31322 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "09198cb684a74ac7b67d55eed7261f41"
format_stamp: "Formatted at 2026-08-06 03:54:02 on dist-test-slave-bqcl"
server_key: "939752c11d58bf1396ee46c90a20931b"
server_key_iv: "342230c665987b19da85d94869a2fe3f"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:02.443883 31322 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260806 03:54:02.458196 31347 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:02.458436 31322 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:54:02.458544 31322 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "09198cb684a74ac7b67d55eed7261f41"
format_stamp: "Formatted at 2026-08-06 03:54:02 on dist-test-slave-bqcl"
server_key: "939752c11d58bf1396ee46c90a20931b"
server_key_iv: "342230c665987b19da85d94869a2fe3f"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:02.458639 31322 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:02.470930 31322 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:02.472501 31322 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:02.472630 31322 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:02.476620 31322 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:37275
I20260806 03:54:02.476679 31442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:37275 every 8 connection(s)
I20260806 03:54:02.477015 31322 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:02.477663 31444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:02.480074 31444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Bootstrap starting.
I20260806 03:54:02.480641 31444 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:02.480914 31444 log.cc:824] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:02.481559 31444 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: No bootstrap required, opened a new log
I20260806 03:54:02.483610 31444 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } }
I20260806 03:54:02.483768 31444 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:02.483803 31444 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09198cb684a74ac7b67d55eed7261f41, State: Initialized, Role: FOLLOWER
I20260806 03:54:02.483947 31444 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } }
I20260806 03:54:02.484007 31444 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:54:02.484033 31444 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:54:02.484081 31444 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:02.484210 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 31322
I20260806 03:54:02.484300 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:54:02.484467 12349 external_mini_cluster.cc:1428] Setting key b9bd78eb37729539bcc46ce3200ab931
I20260806 03:54:02.484690 31444 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } }
I20260806 03:54:02.484799 31444 leader_election.cc:302] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09198cb684a74ac7b67d55eed7261f41; no voters:
I20260806 03:54:02.485172 31444 leader_election.cc:288] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:54:02.485212 31451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:02.485396 31451 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 1 LEADER]: Becoming Leader. State: Replica: 09198cb684a74ac7b67d55eed7261f41, State: Running, Role: LEADER
I20260806 03:54:02.485443 31444 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:02.486253 31451 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } }
I20260806 03:54:02.487116 31452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09198cb684a74ac7b67d55eed7261f41. Latest consensus state: current_term: 1 leader_uuid: "09198cb684a74ac7b67d55eed7261f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } } }
I20260806 03:54:02.487123 31453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "09198cb684a74ac7b67d55eed7261f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } } }
I20260806 03:54:02.487267 31452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:02.487288 31444 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 03:54:02.487303 31453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:02.490737 31473 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:02.491375 31473 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 03:54:02 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:02.494554 31473 catalog_manager.cc:1347] Generated new cluster ID: a82b508abb3c4990a99c3b61496b173c
I20260806 03:54:02.494628 31473 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:54:02.498844 31450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:02.484908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54800 (local address 127.12.15.126:37275)
0806 03:54:02.485060 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:02.485071 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:02.485109 (+ 38us) server_negotiation.cc:407] Connection header received
0806 03:54:02.485258 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:02.485265 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:02.485357 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:02.486320 (+ 963us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:02.486427 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:02.489848 (+ 3421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:02.490892 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:02.491488 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:02.491921 (+ 433us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:02.494517 (+ 2596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:02.494521 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:02.494522 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:02.494534 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:02.497153 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:02.497553 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:02.497557 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:02.497557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:02.497677 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:02.497891 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:02.497892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:02.497892 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:02.498148 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:02.498297 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:02.498522 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 03:54:02.498647 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107,"thread_start_us":76,"threads_started":1}
I20260806 03:54:02.501967 31473 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:54:02.502513 31473 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:54:02.508163 31473 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Generated new TSK 0
I20260806 03:54:02.508328 31473 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:54:03.226609 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:56591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37275
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:03.313328 31520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:03.313537 31520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:03.313570 31520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:03.314716 31520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:03.314759 31520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:03.314795 31520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:54:03.316267 31520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.12.15.84:56591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:03.316499 31520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:03.316722 31520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:03.322659 31531 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:03.322813 31527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:03.322669 31528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:03.323683 31520 server_base.cc:1034] running on GCE node
I20260806 03:54:03.323922 31520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:03.324201 31520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:03.325413 31520 hybrid_clock.cc:648] HybridClock initialized: now 1785988443325392 us; error 36 us; skew 500 ppm
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:03.328673 31520 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:54:03.329237 31520 webserver.cc:460] Webserver started at http://127.12.15.65:43895/ using document root <none> and password file <none>
I20260806 03:54:03.329521 31520 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:03.329624 31520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:03.329778 31520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:03.344823 31520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "b1aa9e0452af4583aa7256b4b66a0959"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "8c2abc9ac28be907d66d20a8385cf7c4"
server_key_iv: "315a64f2da0a61ffef88963fec075400"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.345436 31520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "b1aa9e0452af4583aa7256b4b66a0959"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "8c2abc9ac28be907d66d20a8385cf7c4"
server_key_iv: "315a64f2da0a61ffef88963fec075400"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.347642 31520 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260806 03:54:03.361647 31540 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.361864 31520 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:54:03.361968 31520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "b1aa9e0452af4583aa7256b4b66a0959"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "8c2abc9ac28be907d66d20a8385cf7c4"
server_key_iv: "315a64f2da0a61ffef88963fec075400"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.362066 31520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:03.379392 31520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:03.382064 31520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:03.382323 31520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:03.382622 31520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:03.383052 31520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:03.383104 31520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.383133 31520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:03.383149 31520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.390168 31520 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:36485
I20260806 03:54:03.390681 31520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:54:03.393503 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 31520
I20260806 03:54:03.393612 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:54:03.393800 12349 external_mini_cluster.cc:1428] Setting key a60096b0e8a1c32dfc470a821276ddee
I20260806 03:54:03.402276 31723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:36485 every 8 connection(s)
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
I20260806 03:54:03.412240 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:03.392105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:45399 (local address 127.12.15.126:37275)
0806 03:54:03.402262 (+ 10157us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:03.402264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:03.402275 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:54:03.402302 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:03.402303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:03.402317 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:03.402406 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:03.403197 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.404017 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.405571 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.405874 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.405942 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:03.408564 (+ 2622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:03.408566 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:03.408566 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:03.408574 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:03.410541 (+ 1967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:03.411045 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:03.411047 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:03.411047 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:03.411117 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:03.411431 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:03.411432 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:03.411432 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:03.411582 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:03.411661 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:03.412025 (+ 364us) server_negotiation.cc:299] Negotiation successful
0806 03:54:03.412122 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10120,"thread_start_us":76,"threads_started":1}
I20260806 03:54:03.412415 31727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:03.391805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37275 (local address 127.12.15.65:45399)
0806 03:54:03.398281 (+ 6476us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:03.398338 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:03.398977 (+ 639us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:03.402508 (+ 3531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:03.402514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:03.402795 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:03.403082 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:03.403088 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.404125 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.404126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:03.405463 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:03.405468 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.406025 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.406026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:03.406217 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:03.406829 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:03.406840 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:03.408458 (+ 1618us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:03.410633 (+ 2175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:03.410641 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:03.410644 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:03.410956 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:03.411188 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:03.411189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:03.411189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:03.411350 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:03.411669 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:03.411677 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:03.411854 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 03:54:03.411977 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 03:54:03.412172 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6422,"thread_start_us":82,"threads_started":1}
I20260806 03:54:03.413014 31725 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37275
I20260806 03:54:03.413118 31725 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:03.413339 31725 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:03.413929 31389 ts_manager.cc:194] Registered new tserver with Master: b1aa9e0452af4583aa7256b4b66a0959 (127.12.15.65:36485)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
I20260806 03:54:03.414871 31389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:45399
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:54:03.431609 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:56591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37275
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:03.551164 31733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:03.551448 31733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:03.551487 31733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:03.553308 31733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:03.553373 31733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:03.553412 31733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:54:03.555521 31733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.12.15.84:56591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31733
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:03.555800 31733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:03.556139 31733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:03.562307 31741 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:03.562402 31743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:03.562644 31740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:03.563356 31733 server_base.cc:1034] running on GCE node
I20260806 03:54:03.563582 31733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:03.563866 31733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:03.565042 31733 hybrid_clock.cc:648] HybridClock initialized: now 1785988443565013 us; error 43 us; skew 500 ppm
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:03.568423 31733 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:54:03.568984 31733 webserver.cc:460] Webserver started at http://127.12.15.66:43507/ using document root <none> and password file <none>
I20260806 03:54:03.569269 31733 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:03.569334 31733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:03.569473 31733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:03.582620 31733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "e358e27a72d24c98ea76f61c797ae36f"
server_key_iv: "df89a517359fb5f3654678f85804e370"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.583045 31733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "e358e27a72d24c98ea76f61c797ae36f"
server_key_iv: "df89a517359fb5f3654678f85804e370"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.590473 31733 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.002s sys 0.000s
I20260806 03:54:03.605062 31757 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.610328 31733 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260806 03:54:03.610459 31733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "e358e27a72d24c98ea76f61c797ae36f"
server_key_iv: "df89a517359fb5f3654678f85804e370"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.610565 31733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:03.623528 31733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:03.625317 31733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:03.625448 31733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:03.625669 31733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:03.625994 31733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:03.626035 31733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.626072 31733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:03.626096 31733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.632750 31733 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:40761
I20260806 03:54:03.633198 31733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:54:03.633466 31920 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:40761 every 8 connection(s)
I20260806 03:54:03.638913 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 31733
I20260806 03:54:03.639021 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:54:03.639201 12349 external_mini_cluster.cc:1428] Setting key c972c85058f866b2c05cdc365350c945
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:03.647790 31924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:03.634316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37275 (local address 127.12.15.66:41191)
0806 03:54:03.636807 (+ 2491us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:03.636857 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:03.637521 (+ 664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:03.637839 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:03.637843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:03.638083 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:03.638387 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:03.638395 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.639436 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.639437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:03.641477 (+ 2040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:03.641480 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.642228 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.642229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:03.642370 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:03.642944 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:03.642954 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:03.644330 (+ 1376us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:03.646091 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:03.646098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:03.646100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:03.646419 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:03.646635 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:03.646636 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:03.646636 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:03.646779 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:03.647087 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:03.647094 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:03.647260 (+ 166us) client_negotiation.cc:769] Sending connection context
0806 03:54:03.647366 (+ 106us) client_negotiation.cc:240] Negotiation successful
0806 03:54:03.647526 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2447,"thread_start_us":84,"threads_started":1}
I20260806 03:54:03.648497 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:03.634310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:41191 (local address 127.12.15.126:37275)
0806 03:54:03.634356 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:03.634358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:03.636955 (+ 2597us) server_negotiation.cc:407] Connection header received
0806 03:54:03.637655 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:03.637656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:03.637674 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:03.637749 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:03.638485 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.639319 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.641580 (+ 2261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.641901 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.641941 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:03.644415 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:03.644417 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:03.644417 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:03.644424 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:03.645997 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:03.646506 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:03.646507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:03.646508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:03.646569 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:03.646857 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:03.646858 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:03.646858 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:03.646998 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:03.647056 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:03.648308 (+ 1252us) server_negotiation.cc:299] Negotiation successful
0806 03:54:03.648402 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:54:03.649118 31921 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37275
I20260806 03:54:03.649215 31921 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:03.649410 31921 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:03.649868 31389 ts_manager.cc:194] Registered new tserver with Master: 6ac4efcaabb5472c9c144b0d88d6d5d1 (127.12.15.66:40761)
I20260806 03:54:03.650375 31389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:41191
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:54:03.670692 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:56591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37275
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:03.789713 31928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:03.790053 31928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:03.790105 31928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:03.791777 31928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:03.791834 31928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:03.791872 31928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:54:03.794081 31928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.12.15.84:56591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:03.794354 31928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:03.794596 31928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:03.800631 31935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:03.800855 31928 server_base.cc:1034] running on GCE node
W20260806 03:54:03.800645 31938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:03.801064 31934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:03.801270 31928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:03.801530 31928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:03.802685 31928 hybrid_clock.cc:648] HybridClock initialized: now 1785988443802664 us; error 43 us; skew 500 ppm
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:03.805207 31928 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:54:03.805601 31928 webserver.cc:460] Webserver started at http://127.12.15.67:40749/ using document root <none> and password file <none>
I20260806 03:54:03.805796 31928 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:03.805842 31928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:03.805974 31928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:03.818863 31928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "20a4a4506e644957b1b6f7ccb37f3f1c"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "31b206dcd1e931f155372ee488cd8f4f"
server_key_iv: "73b0d1079b55695ee1544b810a7ced52"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.819299 31928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "20a4a4506e644957b1b6f7ccb37f3f1c"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "31b206dcd1e931f155372ee488cd8f4f"
server_key_iv: "73b0d1079b55695ee1544b810a7ced52"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.822588 31928 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260806 03:54:03.846751 31948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.846925 31928 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260806 03:54:03.847025 31928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "20a4a4506e644957b1b6f7ccb37f3f1c"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "31b206dcd1e931f155372ee488cd8f4f"
server_key_iv: "73b0d1079b55695ee1544b810a7ced52"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:03.847119 31928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:03.864226 31928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:03.866731 31928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:03.866882 31928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:03.867132 31928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:03.867523 31928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:03.867575 31928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.867610 31928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:03.867627 31928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:03.876396 31928 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:39437
I20260806 03:54:03.876828 31928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:54:03.877430 32146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:39437 every 8 connection(s)
I20260806 03:54:03.877449 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 31928
I20260806 03:54:03.877559 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:54:03.877753 12349 external_mini_cluster.cc:1428] Setting key 1b982cf6fbc31bdb7f1d04cea2e7a565
Aug 06 03:54:03 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988443, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:03.892364 31728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:03.878228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:48597 (local address 127.12.15.126:37275)
0806 03:54:03.878269 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:03.878271 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:03.878630 (+ 359us) server_negotiation.cc:407] Connection header received
0806 03:54:03.879302 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:03.879303 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:03.879315 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:03.879343 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:03.880162 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.880888 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.882495 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.882809 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.882879 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:03.885508 (+ 2629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:03.885509 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:03.885510 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:03.885517 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:03.887321 (+ 1804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:03.891158 (+ 3837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:03.891160 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:03.891160 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:03.891229 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:03.891574 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:03.891576 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:03.891576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:03.891724 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:03.891782 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:03.892170 (+ 388us) server_negotiation.cc:299] Negotiation successful
0806 03:54:03.892263 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:54:03.892768 32152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:03.877987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37275 (local address 127.12.15.67:48597)
0806 03:54:03.878516 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:03.878556 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:03.879187 (+ 631us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:03.879520 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:03.879525 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:03.879753 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:03.880041 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:03.880048 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.881020 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.881021 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:03.882381 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:03.882384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:03.882930 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:03.882931 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:03.883079 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:03.883661 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:03.883671 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:03.885405 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:03.890741 (+ 5336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:03.890748 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:03.890751 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:03.891062 (+ 311us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:03.891305 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:03.891306 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:03.891307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:03.891456 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:03.891817 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:03.891824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:03.891983 (+ 159us) client_negotiation.cc:769] Sending connection context
0806 03:54:03.892105 (+ 122us) client_negotiation.cc:240] Negotiation successful
0806 03:54:03.892276 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":76,"threads_started":1}
I20260806 03:54:03.893551 32148 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37275
I20260806 03:54:03.893715 32148 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:03.893968 32148 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:03.894430 31389 ts_manager.cc:194] Registered new tserver with Master: 20a4a4506e644957b1b6f7ccb37f3f1c (127.12.15.67:39437)
I20260806 03:54:03.894915 31389 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:48597
I20260806 03:54:03.900352 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:54:03.901101 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 31520
I20260806 03:54:03.950073 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 31733
I20260806 03:54:03.969970 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 31928
I20260806 03:54:03.977329 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 31322
I20260806 03:54:03.993477 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37275
--webserver_interface=127.12.15.126
--webserver_port=33205
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:37275
--ranger_config_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:56591/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.15.84:56591/kms with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:04.074292 32154 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:04.074491 32154 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:04.074523 32154 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:04.075644 32154 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:04.075685 32154 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:04.075703 32154 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:04.075721 32154 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:04.075734 32154 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:04.077277 32154 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.15.84:56591/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:37275
--ranger_config_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=33205
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32154
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:04.077519 32154 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:04.077723 32154 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:04.082409 32161 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:04.082423 32162 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:04.082409 32166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:04.082705 32154 server_base.cc:1034] running on GCE node
I20260806 03:54:04.082942 32154 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:04.083225 32154 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:04.084354 32154 hybrid_clock.cc:648] HybridClock initialized: now 1785988444084341 us; error 28 us; skew 500 ppm
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:04.086782 32154 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:04.087189 32154 webserver.cc:460] Webserver started at http://127.12.15.126:33205/ using document root <none> and password file <none>
I20260806 03:54:04.087386 32154 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:04.087441 32154 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:04.123582 32154 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:04.124398 32181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.124580 32154 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:04.124678 32154 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "09198cb684a74ac7b67d55eed7261f41"
format_stamp: "Formatted at 2026-08-06 03:54:02 on dist-test-slave-bqcl"
server_key: "939752c11d58bf1396ee46c90a20931b"
server_key_iv: "342230c665987b19da85d94869a2fe3f"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:04.125069 32154 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:04.138492 32154 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:04.140374 32154 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:04.140501 32154 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:04.144608 32154 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:37275
I20260806 03:54:04.144681 32258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:37275 every 8 connection(s)
I20260806 03:54:04.145018 32154 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:04.146324 32260 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:54:04.146926 32260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Bootstrap starting.
I20260806 03:54:04.149385 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 32154
I20260806 03:54:04.149657 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:56591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:36485
--local_ip_for_outbound_sockets=127.12.15.65
--tserver_master_addrs=127.12.15.126:37275
--webserver_port=43895
--webserver_interface=127.12.15.65
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.15.84:56591/kms with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:04.155072 32260 log.cc:824] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:04.157387 32260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:54:04.157706 32260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Bootstrap complete.
I20260806 03:54:04.160974 32260 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } }
I20260806 03:54:04.161304 32260 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09198cb684a74ac7b67d55eed7261f41, State: Initialized, Role: FOLLOWER
I20260806 03:54:04.161495 32260 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } }
I20260806 03:54:04.161587 32260 raft_consensus.cc:388] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:54:04.161630 32260 raft_consensus.cc:482] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:54:04.161684 32260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:54:04.162978 32260 raft_consensus.cc:504] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } }
I20260806 03:54:04.163086 32260 leader_election.cc:302] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 09198cb684a74ac7b67d55eed7261f41; no voters:
I20260806 03:54:04.163259 32260 leader_election.cc:288] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:54:04.163319 32265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:54:04.163472 32265 raft_consensus.cc:686] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [term 2 LEADER]: Becoming Leader. State: Replica: 09198cb684a74ac7b67d55eed7261f41, State: Running, Role: LEADER
I20260806 03:54:04.163491 32260 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:04.163574 32265 consensus_queue.cc:237] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } }
I20260806 03:54:04.163816 32265 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 09198cb684a74ac7b67d55eed7261f41. Latest consensus state: current_term: 2 leader_uuid: "09198cb684a74ac7b67d55eed7261f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } } }
I20260806 03:54:04.163880 32265 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:04.163890 32266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "09198cb684a74ac7b67d55eed7261f41" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "09198cb684a74ac7b67d55eed7261f41" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37275 } } }
I20260806 03:54:04.163977 32266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41 [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:04.164434 32274 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:04.164976 32274 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:54:04.165112 32274 catalog_manager.cc:1259] Loaded cluster ID: a82b508abb3c4990a99c3b61496b173c
I20260806 03:54:04.165150 32274 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:54:04.165431 32260 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260806 03:54:04.166368 32274 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:54:04.166543 32274 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Loaded TSK: 0
I20260806 03:54:04.166764 32274 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:54:04.267203 32263 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:04.267517 32263 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:04.267561 32263 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:04.269254 32263 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:04.269317 32263 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:04.269357 32263 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:54:04.271646 32263 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.15.84:56591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:36485
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=43895
--tserver_master_addrs=127.12.15.126:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32263
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:04.271906 32263 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:04.272207 32263 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:04.278911 32318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:04.279106 32322 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:04.279282 32263 server_base.cc:1034] running on GCE node
W20260806 03:54:04.279313 32319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:04.279664 32263 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:04.279959 32263 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:04.281136 32263 hybrid_clock.cc:648] HybridClock initialized: now 1785988444281109 us; error 41 us; skew 500 ppm
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:04.284353 32263 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:54:04.284977 32263 webserver.cc:460] Webserver started at http://127.12.15.65:43895/ using document root <none> and password file <none>
I20260806 03:54:04.285292 32263 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:04.285429 32263 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:04.314324 32263 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260806 03:54:04.330780 32332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.331043 32263 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.001s sys 0.000s
I20260806 03:54:04.331141 32263 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "b1aa9e0452af4583aa7256b4b66a0959"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "8c2abc9ac28be907d66d20a8385cf7c4"
server_key_iv: "315a64f2da0a61ffef88963fec075400"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:04.331634 32263 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:04.368263 32263 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:04.370853 32263 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:04.371083 32263 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:04.371346 32263 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:04.371798 32263 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:04.371884 32263 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.371948 32263 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:04.371989 32263 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.378430 32263 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:36485
I20260806 03:54:04.378964 32263 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:54:04.388537 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 32263
I20260806 03:54:04.388885 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:56591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:40761
--local_ip_for_outbound_sockets=127.12.15.66
--tserver_master_addrs=127.12.15.126:37275
--webserver_port=43507
--webserver_interface=127.12.15.66
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.15.84:56591/kms with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:04.390853 32496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:36485 every 8 connection(s)
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:04.418563 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:04.391960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37275 (local address 127.12.15.65:59715)
0806 03:54:04.393835 (+ 1875us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:04.393892 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:04.394546 (+ 654us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:04.395399 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:04.395403 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:04.395659 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:04.395958 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:04.395966 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.397672 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.397673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:04.398982 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:04.398986 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.399967 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.399968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:04.400118 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:04.400757 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:04.400767 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:04.405349 (+ 4582us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:04.416276 (+ 10927us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:04.416288 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:04.416291 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:04.416636 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:04.417017 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:04.417018 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:04.417019 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:04.417187 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:04.417775 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:04.417783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:04.417958 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 03:54:04.418063 (+ 105us) client_negotiation.cc:240] Negotiation successful
0806 03:54:04.418289 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1818,"thread_start_us":94,"threads_started":1}
I20260806 03:54:04.419656 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:04.392369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:59715 (local address 127.12.15.126:37275)
0806 03:54:04.394778 (+ 2409us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:04.394789 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:04.394828 (+ 39us) server_negotiation.cc:407] Connection header received
0806 03:54:04.394985 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:04.394992 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:04.395070 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:04.395213 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:04.396108 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.397544 (+ 1436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.399095 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.399727 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.399835 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:04.405491 (+ 5656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:04.405497 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:04.405497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:04.405512 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:04.416153 (+ 10641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:04.416776 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:04.416781 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:04.416782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:04.416924 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:04.417281 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:04.417282 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:04.417283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:04.417641 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:04.417742 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:04.419270 (+ 1528us) server_negotiation.cc:299] Negotiation successful
0806 03:54:04.419438 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2346,"thread_start_us":90,"threads_started":1}
I20260806 03:54:04.420742 32499 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37275
I20260806 03:54:04.420917 32499 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.421218 32499 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
W20260806 03:54:04.422247 32499 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:37275 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 03:54:04.422410 32499 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.422531 32499 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:04.422902 32499 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.423103 32499 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
W20260806 03:54:04.423457 32499 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:54:04.542435 32503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:04.542855 32503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:04.542971 32503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:04.544749 32503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:04.544857 32503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:04.544951 32503 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:54:04.547374 32503 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.15.84:56591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:40761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=43507
--tserver_master_addrs=127.12.15.126:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32503
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:04.547849 32503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:04.548228 32503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:04.555155 32520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:04.555253 32516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:04.555591 32515 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:04.556093 32503 server_base.cc:1034] running on GCE node
I20260806 03:54:04.556329 32503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:04.556617 32503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:04.557794 32503 hybrid_clock.cc:648] HybridClock initialized: now 1785988444557773 us; error 36 us; skew 500 ppm
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:04.561092 32503 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:54:04.561689 32503 webserver.cc:460] Webserver started at http://127.12.15.66:43507/ using document root <none> and password file <none>
I20260806 03:54:04.561985 32503 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:04.562072 32503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:04.583877 32503 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260806 03:54:04.590700 32529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.591040 32503 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.000s sys 0.000s
I20260806 03:54:04.591221 32503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "e358e27a72d24c98ea76f61c797ae36f"
server_key_iv: "df89a517359fb5f3654678f85804e370"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:04.591665 32503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:04.611910 32503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:04.614439 32503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:04.614689 32503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:04.614995 32503 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:04.615464 32503 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:04.615581 32503 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.615664 32503 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:04.615727 32503 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.624583 32503 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:40761
I20260806 03:54:04.625170 32503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:54:04.627125 32706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:40761 every 8 connection(s)
I20260806 03:54:04.632045 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 32503
I20260806 03:54:04.632362 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.12.15.84:56591/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:39437
--local_ip_for_outbound_sockets=127.12.15.67
--tserver_master_addrs=127.12.15.126:37275
--webserver_port=40749
--webserver_interface=127.12.15.67
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:44453
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.12.15.84:56591/kms with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:04.638322 32710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:04.626643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37275 (local address 127.12.15.66:49871)
0806 03:54:04.627345 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:04.627388 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:04.628071 (+ 683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:04.628463 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:04.628468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:04.628715 (+ 247us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:04.629015 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:04.629023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.630059 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.630060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:04.631387 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:04.631390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.631960 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.631961 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:04.632103 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:04.632953 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:04.632964 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:04.634551 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:04.636518 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:04.636525 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:04.636527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:04.636833 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:04.637074 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:04.637075 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:04.637075 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:04.637231 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:04.637545 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:04.637552 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:04.637729 (+ 177us) client_negotiation.cc:769] Sending connection context
0806 03:54:04.637844 (+ 115us) client_negotiation.cc:240] Negotiation successful
0806 03:54:04.638008 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":100,"threads_started":1}
I20260806 03:54:04.638825 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:04.626963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:49871 (local address 127.12.15.126:37275)
0806 03:54:04.627017 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:04.627019 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:04.627478 (+ 459us) server_negotiation.cc:407] Connection header received
0806 03:54:04.628213 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:04.628215 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:04.628231 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:04.628263 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:04.629153 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.629924 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.631490 (+ 1566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.631792 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.631840 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:04.634664 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:04.634666 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:04.634666 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:04.634674 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:04.636435 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:04.636928 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:04.636929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:04.636930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:04.636999 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:04.637308 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:04.637310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:04.637311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:04.637457 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:04.637517 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:04.638634 (+ 1117us) server_negotiation.cc:299] Negotiation successful
0806 03:54:04.638728 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:54:04.639532 32707 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37275
I20260806 03:54:04.639635 32707 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.639832 32707 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
W20260806 03:54:04.640515 32707 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:37275 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 03:54:04.640581 32707 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.640640 32707 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:04.640962 32707 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.641031 32707 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
W20260806 03:54:04.641343 32707 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:54:04.751294 32711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:04.751644 32711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:04.751757 32711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:04.753489 32711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:04.753607 32711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:04.753691 32711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:54:04.755846 32711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:44453
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.12.15.84:56591/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:39437
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=40749
--tserver_master_addrs=127.12.15.126:37275
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:04.756328 32711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:04.756667 32711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:54:04.763484 32711 server_base.cc:1034] running on GCE node
W20260806 03:54:04.764194 32725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:04.764571 32723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:04.764933 32722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:04.765363 32711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:04.765654 32711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:04.770217 32711 hybrid_clock.cc:648] HybridClock initialized: now 1785988444770189 us; error 39 us; skew 500 ppm
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:04.773438 32711 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:54:04.773962 32711 webserver.cc:460] Webserver started at http://127.12.15.67:40749/ using document root <none> and password file <none>
I20260806 03:54:04.774266 32711 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:04.774333 32711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:04.794003 32711 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.001s sys 0.000s
I20260806 03:54:04.801995 32740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.802402 32711 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.000s sys 0.000s
I20260806 03:54:04.802583 32711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "20a4a4506e644957b1b6f7ccb37f3f1c"
format_stamp: "Formatted at 2026-08-06 03:54:03 on dist-test-slave-bqcl"
server_key: "31b206dcd1e931f155372ee488cd8f4f"
server_key_iv: "73b0d1079b55695ee1544b810a7ced52"
server_key_version: "kuduclusterkey@0"
I20260806 03:54:04.802994 32711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:04.823661 32711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:04.826192 32711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:04.826411 32711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:04.826699 32711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:04.827111 32711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:04.827199 32711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.827273 32711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:04.827330 32711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:04.833570 32711 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:39437
I20260806 03:54:04.834098 32711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:54:04.836702 431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:39437 every 8 connection(s)
Aug 06 03:54:04 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988444, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:04.844379 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 32711
I20260806 03:54:04.848460 32507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:04.837594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:52103 (local address 127.12.15.126:37275)
0806 03:54:04.837640 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:04.837642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:04.838169 (+ 527us) server_negotiation.cc:407] Connection header received
0806 03:54:04.838855 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:04.838857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:04.838870 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:04.838902 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:04.839722 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.840508 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.841999 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.842352 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.842396 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:04.845292 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:04.845293 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:04.845293 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:04.845301 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:04.846934 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:04.847429 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:04.847430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:04.847431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:04.847486 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:04.847762 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:04.847763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:04.847763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:04.847887 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:04.847929 (+ 42us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:04.848289 (+ 360us) server_negotiation.cc:299] Negotiation successful
0806 03:54:04.848366 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:54:04.849169 435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:04.837302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37275 (local address 127.12.15.67:52103)
0806 03:54:04.837899 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:04.838060 (+ 161us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:04.838721 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:04.839110 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:04.839115 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:04.839338 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:04.839612 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:04.839619 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.840623 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.840624 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:04.841888 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:04.841891 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.842508 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.842509 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:04.842646 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:04.843188 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:04.843198 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:04.845133 (+ 1935us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:04.847073 (+ 1940us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:04.847079 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:04.847081 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:04.847348 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:04.847555 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:04.847556 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:04.847556 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:04.847685 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:04.847975 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:04.847981 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:04.848144 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 03:54:04.848775 (+ 631us) client_negotiation.cc:240] Negotiation successful
0806 03:54:04.848933 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":77,"threads_started":1}
I20260806 03:54:04.849776 432 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37275
I20260806 03:54:04.849864 432 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.850054 432 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
W20260806 03:54:04.850703 432 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:37275 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260806 03:54:04.850776 432 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.850838 432 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:04.851197 432 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:04.851270 432 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
W20260806 03:54:04.851608 432 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:54:04.856832 438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:04.845455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54808 (local address 127.12.15.126:37275)
0806 03:54:04.845912 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:04.845914 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:04.845921 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:54:04.845938 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:04.845939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:04.845948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:04.846004 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:04.846805 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.847505 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.850935 (+ 3430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:04.851205 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:04.851261 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:04.854226 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:04.854228 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:04.854229 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:04.854236 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:04.855765 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:04.856136 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:04.856137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:04.856137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:04.856194 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:04.856354 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:04.856354 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:04.856355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:04.856459 (+ 104us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:04.856618 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:04.856658 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 03:54:04.856736 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":83,"threads_started":1}
Aug 06 03:54:05 dist-test-slave-bqcl krb5kdc[30607](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1785988445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:05.423686 32499 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:05.423857 32499 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:05.424263 32205 ts_manager.cc:194] Registered new tserver with Master: b1aa9e0452af4583aa7256b4b66a0959 (127.12.15.65:36485)
I20260806 03:54:05.425303 32205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:59715
I20260806 03:54:05.641532 32707 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:05.641695 32707 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:05.641959 32205 ts_manager.cc:194] Registered new tserver with Master: 6ac4efcaabb5472c9c144b0d88d6d5d1 (127.12.15.66:40761)
I20260806 03:54:05.642524 32205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:49871
Aug 06 03:54:05 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:05.851783 432 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:05.851943 432 heartbeater.cc:511] Master 127.12.15.126:37275 requested a full tablet report, sending...
I20260806 03:54:05.852273 32205 ts_manager.cc:194] Registered new tserver with Master: 20a4a4506e644957b1b6f7ccb37f3f1c (127.12.15.67:39437)
I20260806 03:54:05.852782 32205 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:52103
I20260806 03:54:05.855034 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:54:05.866290 462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.855940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57870 (local address 127.12.15.126:37275)
0806 03:54:05.858460 (+ 2520us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:05.858462 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:05.858468 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:54:05.858488 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:05.858489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:05.858500 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:05.858556 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:05.859241 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.859961 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.861006 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.861313 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.861382 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.862967 (+ 1585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:05.862968 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:05.862969 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:05.862976 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:05.864821 (+ 1845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:05.865259 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:05.865261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:05.865261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:05.865329 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:05.865555 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:05.865556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:05.865556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:05.865696 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:05.865797 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:05.865949 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 03:54:05.866049 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2501,"thread_start_us":71,"threads_started":1}
I20260806 03:54:05.868369 32205 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57870:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:54:05.870709 32205 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:54:05.882683 474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.874757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:39437 (local address 127.0.0.1:54266)
0806 03:54:05.878111 (+ 3354us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:05.878153 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:05.878208 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:05.878505 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:05.878509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:05.878511 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:05.878577 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.878579 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.879819 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.879820 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.881073 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.881077 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.882054 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.882055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.882425 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.882449 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 03:54:05.882511 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:54:05.882563 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3337,"thread_start_us":57,"threads_started":1}
I20260806 03:54:05.883370 478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.874858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54266 (local address 127.12.15.67:39437)
0806 03:54:05.878035 (+ 3177us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:05.878039 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:05.878265 (+ 226us) server_negotiation.cc:407] Connection header received
0806 03:54:05.878306 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:05.878308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:05.878325 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:05.878393 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:05.878661 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.879723 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.881168 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.881885 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.881950 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.882018 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:05.883250 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0806 03:54:05.883295 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3144,"thread_start_us":48,"threads_started":1}
I20260806 03:54:05.884227 477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.874914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36485 (local address 127.0.0.1:53168)
0806 03:54:05.879860 (+ 4946us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:05.879870 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:05.879920 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:05.880200 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:05.880201 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:05.880203 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:05.880245 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.880246 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.881467 (+ 1221us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.881469 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.882578 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.882581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.883999 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.884000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.884064 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.884069 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:54:05.884114 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:54:05.884154 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4927,"thread_start_us":68,"threads_started":1}
I20260806 03:54:05.884254 339 tablet_service.cc:1467] Processing CreateTablet for tablet 18e3e3fff8094a2dbea19994119ac5ca (DEFAULT_TABLE table=test-table [id=1778cf78c2514a8392a32c54805cba8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:05.884600 483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.875176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53168 (local address 127.12.15.65:36485)
0806 03:54:05.877942 (+ 2766us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:05.877947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:05.879981 (+ 2034us) server_negotiation.cc:407] Connection header received
0806 03:54:05.880021 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:05.880024 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:05.880040 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:05.880110 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:05.880316 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.881370 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.882679 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.883403 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.883480 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.883549 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:05.884469 (+ 920us) server_negotiation.cc:299] Negotiation successful
0806 03:54:05.884523 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2729,"thread_start_us":57,"threads_started":1}
I20260806 03:54:05.884637 339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18e3e3fff8094a2dbea19994119ac5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:05.886766 32392 tablet_service.cc:1467] Processing CreateTablet for tablet 18e3e3fff8094a2dbea19994119ac5ca (DEFAULT_TABLE table=test-table [id=1778cf78c2514a8392a32c54805cba8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:05.887079 32392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18e3e3fff8094a2dbea19994119ac5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:05.888062 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.874645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38228 (local address 127.12.15.66:40761)
0806 03:54:05.883844 (+ 9199us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:05.883849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:05.883859 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:54:05.883891 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:05.883893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:05.883907 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:05.884887 (+ 980us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:05.884970 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.885990 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.887187 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.887667 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.887729 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.887784 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:05.887919 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 03:54:05.887964 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9162,"thread_start_us":65,"threads_started":1}
I20260806 03:54:05.888285 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.874544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:40761 (local address 127.0.0.1:38228)
0806 03:54:05.877680 (+ 3136us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:05.877725 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:05.877798 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:05.884762 (+ 6964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:05.884763 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:05.884765 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:05.884804 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.884805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.886431 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.886432 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.887105 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.887107 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.887760 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.887761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.887820 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.887823 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:54:05.888174 (+ 351us) client_negotiation.cc:240] Negotiation successful
0806 03:54:05.888219 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3101,"thread_start_us":78,"threads_started":1}
I20260806 03:54:05.889061 32603 tablet_service.cc:1467] Processing CreateTablet for tablet 18e3e3fff8094a2dbea19994119ac5ca (DEFAULT_TABLE table=test-table [id=1778cf78c2514a8392a32c54805cba8f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:05.889384 32603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18e3e3fff8094a2dbea19994119ac5ca. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:05.890698 488 tablet_bootstrap.cc:492] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959: Bootstrap starting.
I20260806 03:54:05.891396 488 tablet_bootstrap.cc:654] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:05.891702 488 log.cc:824] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:05.892437 488 tablet_bootstrap.cc:492] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959: No bootstrap required, opened a new log
I20260806 03:54:05.892510 488 ts_tablet_manager.cc:1397] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:54:05.894729 488 raft_consensus.cc:348] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.894891 488 raft_consensus.cc:374] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:05.894920 488 raft_consensus.cc:729] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1aa9e0452af4583aa7256b4b66a0959, State: Initialized, Role: FOLLOWER
I20260806 03:54:05.895007 485 tablet_bootstrap.cc:492] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: Bootstrap starting.
I20260806 03:54:05.895021 488 consensus_queue.cc:260] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.895246 488 ts_tablet_manager.cc:1428] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:54:05.895536 32499 heartbeater.cc:503] Master 127.12.15.126:37275 was elected leader, sending a full tablet report...
I20260806 03:54:05.895936 485 tablet_bootstrap.cc:654] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:05.896471 490 tablet_bootstrap.cc:492] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: Bootstrap starting.
I20260806 03:54:05.896806 485 log.cc:824] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:05.897115 490 tablet_bootstrap.cc:654] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:05.897485 490 log.cc:824] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:05.900629 485 tablet_bootstrap.cc:492] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: No bootstrap required, opened a new log
I20260806 03:54:05.900736 485 ts_tablet_manager.cc:1397] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 03:54:05.903268 485 raft_consensus.cc:348] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.903446 485 raft_consensus.cc:374] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:05.903496 485 raft_consensus.cc:729] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20a4a4506e644957b1b6f7ccb37f3f1c, State: Initialized, Role: FOLLOWER
I20260806 03:54:05.903661 490 tablet_bootstrap.cc:492] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: No bootstrap required, opened a new log
I20260806 03:54:05.903618 485 consensus_queue.cc:260] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.903731 490 ts_tablet_manager.cc:1397] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.000s
I20260806 03:54:05.903891 485 ts_tablet_manager.cc:1428] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:54:05.903916 432 heartbeater.cc:503] Master 127.12.15.126:37275 was elected leader, sending a full tablet report...
I20260806 03:54:05.906879 490 raft_consensus.cc:348] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.907055 490 raft_consensus.cc:374] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:05.907161 490 raft_consensus.cc:729] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ac4efcaabb5472c9c144b0d88d6d5d1, State: Initialized, Role: FOLLOWER
I20260806 03:54:05.907348 490 consensus_queue.cc:260] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.907670 490 ts_tablet_manager.cc:1428] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260806 03:54:05.907713 32707 heartbeater.cc:503] Master 127.12.15.126:37275 was elected leader, sending a full tablet report...
Aug 06 03:54:05 dist-test-slave-bqcl krb5kdc[30607](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988445, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for HTTP/127.12.15.84@KRBTEST.COM
I20260806 03:54:05.955124 495 raft_consensus.cc:482] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:05.955296 495 raft_consensus.cc:504] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.955737 495 leader_election.cc:288] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6ac4efcaabb5472c9c144b0d88d6d5d1 (127.12.15.66:40761), b1aa9e0452af4583aa7256b4b66a0959 (127.12.15.65:36485)
I20260806 03:54:05.960618 476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.956154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:37985 (local address 127.12.15.66:40761)
0806 03:54:05.956241 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:05.956242 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:05.956249 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:54:05.956274 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:05.956274 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:05.956282 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:05.956326 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:05.956593 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.957118 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.959614 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.960045 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.960082 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.960090 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:05.960494 (+ 404us) server_negotiation.cc:299] Negotiation successful
0806 03:54:05.960546 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260806 03:54:05.960846 507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.955909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:40761 (local address 127.12.15.67:37985)
0806 03:54:05.956094 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:05.956106 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:05.956159 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:05.956345 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:05.956346 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:05.956348 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:05.956429 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.956431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.957805 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.957806 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.959510 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.959514 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.960329 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.960330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.960401 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.960406 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:54:05.960708 (+ 302us) client_negotiation.cc:240] Negotiation successful
0806 03:54:05.960753 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":122,"threads_started":1}
I20260806 03:54:05.962674 32636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18e3e3fff8094a2dbea19994119ac5ca" candidate_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" is_pre_election: true
I20260806 03:54:05.962849 32636 raft_consensus.cc:2454] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20a4a4506e644957b1b6f7ccb37f3f1c in term 0.
I20260806 03:54:05.963197 32742 leader_election.cc:302] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a4a4506e644957b1b6f7ccb37f3f1c, 6ac4efcaabb5472c9c144b0d88d6d5d1; no voters:
I20260806 03:54:05.963359 495 raft_consensus.cc:2781] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:54:05.963440 495 raft_consensus.cc:482] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:54:05.963462 495 raft_consensus.cc:3037] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:05.964371 495 raft_consensus.cc:504] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.964520 495 leader_election.cc:288] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [CANDIDATE]: Term 1 election: Requested vote from peers 6ac4efcaabb5472c9c144b0d88d6d5d1 (127.12.15.66:40761), b1aa9e0452af4583aa7256b4b66a0959 (127.12.15.65:36485)
I20260806 03:54:05.965832 32636 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18e3e3fff8094a2dbea19994119ac5ca" candidate_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1"
I20260806 03:54:05.965925 32636 raft_consensus.cc:3037] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:05.966969 32636 raft_consensus.cc:2454] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a4a4506e644957b1b6f7ccb37f3f1c in term 1.
I20260806 03:54:05.967200 32742 leader_election.cc:302] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 20a4a4506e644957b1b6f7ccb37f3f1c, 6ac4efcaabb5472c9c144b0d88d6d5d1; no voters:
I20260806 03:54:05.967283 495 raft_consensus.cc:2781] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:05.967413 495 raft_consensus.cc:686] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 1 LEADER]: Becoming Leader. State: Replica: 20a4a4506e644957b1b6f7ccb37f3f1c, State: Running, Role: LEADER
I20260806 03:54:05.967502 495 consensus_queue.cc:237] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } }
I20260806 03:54:05.968713 32204 catalog_manager.cc:5495] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c reported cstate change: term changed from 0 to 1, leader changed from <none> to 20a4a4506e644957b1b6f7ccb37f3f1c (127.12.15.67). New cstate: current_term: 1 leader_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 39437 } health_report { overall_health: HEALTHY } } }
I20260806 03:54:05.971096 483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.956129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:34031 (local address 127.12.15.65:36485)
0806 03:54:05.956208 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:05.956211 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:05.961061 (+ 4850us) server_negotiation.cc:407] Connection header received
0806 03:54:05.961097 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:05.961098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:05.961110 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:05.961163 (+ 53us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:05.961342 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.961988 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.969919 (+ 7931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.970443 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.970483 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.970491 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:05.970980 (+ 489us) server_negotiation.cc:299] Negotiation successful
0806 03:54:05.971037 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260806 03:54:05.971093 510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:05.955945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36485 (local address 127.12.15.67:34031)
0806 03:54:05.960903 (+ 4958us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:05.960986 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:05.961054 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:05.961213 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:05.961214 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:05.961216 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:05.961267 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.961269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.963818 (+ 2549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.963819 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.969797 (+ 5978us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:05.969801 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:05.970830 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:05.970831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:05.970907 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:05.970913 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:54:05.970962 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:54:05.971010 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4935,"thread_start_us":113,"threads_started":1}
I20260806 03:54:05.971446 32433 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18e3e3fff8094a2dbea19994119ac5ca" candidate_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1aa9e0452af4583aa7256b4b66a0959"
I20260806 03:54:05.971443 32406 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18e3e3fff8094a2dbea19994119ac5ca" candidate_uuid: "20a4a4506e644957b1b6f7ccb37f3f1c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1aa9e0452af4583aa7256b4b66a0959" is_pre_election: true
I20260806 03:54:05.971609 32433 raft_consensus.cc:3037] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:05.972436 32433 raft_consensus.cc:2454] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20a4a4506e644957b1b6f7ccb37f3f1c in term 1.
I20260806 03:54:06.010007 478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:06.006480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54274 (local address 127.12.15.67:39437)
0806 03:54:06.006544 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:06.006546 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:06.006559 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:54:06.006608 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:06.006609 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:06.006636 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:06.006729 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:54:06.006913 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:06.007672 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:06.008819 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:06.009145 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:06.009210 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:06.009427 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:54:06.009572 (+ 145us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:54:06.009653 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:06.009902 (+ 249us) server_negotiation.cc:299] Negotiation successful
0806 03:54:06.009925 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:54:06.023213 32636 raft_consensus.cc:1261] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 1 FOLLOWER]: Refusing update from remote peer 20a4a4506e644957b1b6f7ccb37f3f1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:06.023227 32433 raft_consensus.cc:1261] T 18e3e3fff8094a2dbea19994119ac5ca P b1aa9e0452af4583aa7256b4b66a0959 [term 1 FOLLOWER]: Refusing update from remote peer 20a4a4506e644957b1b6f7ccb37f3f1c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:06.023581 495 consensus_queue.cc:1035] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ac4efcaabb5472c9c144b0d88d6d5d1" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 40761 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:06.023715 511 consensus_queue.cc:1035] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1aa9e0452af4583aa7256b4b66a0959" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36485 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:06.036926 514 mvcc.cc:205] Tried to move back new op lower bound from 7315408674908123136 to 7315408674683555840. Current Snapshot: MvccSnapshot[applied={T|T < 7315408674908123136}]
I20260806 03:54:06.038790 519 mvcc.cc:205] Tried to move back new op lower bound from 7315408674908123136 to 7315408674683555840. Current Snapshot: MvccSnapshot[applied={T|T < 7315408674908123136}]
I20260806 03:54:06.048188 515 mvcc.cc:205] Tried to move back new op lower bound from 7315408674908123136 to 7315408674683555840. Current Snapshot: MvccSnapshot[applied={T|T < 7315408674908123136}]
I20260806 03:54:06.050923 32205 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:06.051045 32205 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57870:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:06.051918 32205 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 09198cb684a74ac7b67d55eed7261f41: Sending DeleteTablet for 3 replicas of tablet 18e3e3fff8094a2dbea19994119ac5ca
I20260806 03:54:06.052469 32603 tablet_service.cc:1514] Processing DeleteTablet for tablet 18e3e3fff8094a2dbea19994119ac5ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:06 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:38228
I20260806 03:54:06.052736 339 tablet_service.cc:1514] Processing DeleteTablet for tablet 18e3e3fff8094a2dbea19994119ac5ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:06 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:54266
I20260806 03:54:06.053058 32392 tablet_service.cc:1514] Processing DeleteTablet for tablet 18e3e3fff8094a2dbea19994119ac5ca with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:06 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:53168
I20260806 03:54:06.053989 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 32263
I20260806 03:54:06.059105 527 tablet_replica.cc:330] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: stopping tablet replica
I20260806 03:54:06.059243 527 raft_consensus.cc:2229] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:06.059372 527 raft_consensus.cc:2258] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:06.059558 528 tablet_replica.cc:330] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: stopping tablet replica
I20260806 03:54:06.059648 528 raft_consensus.cc:2229] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:54:06.059746 528 raft_consensus.cc:2258] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:06.060312 528 ts_tablet_manager.cc:1905] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:06.062196 528 ts_tablet_manager.cc:1918] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:06.062304 528 log.cc:1196] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/18e3e3fff8094a2dbea19994119ac5ca
I20260806 03:54:06.062606 528 ts_tablet_manager.cc:1939] T 18e3e3fff8094a2dbea19994119ac5ca P 20a4a4506e644957b1b6f7ccb37f3f1c: Deleting consensus metadata
I20260806 03:54:06.063680 527 ts_tablet_manager.cc:1905] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:06.064045 32183 catalog_manager.cc:4841] TS 20a4a4506e644957b1b6f7ccb37f3f1c (127.12.15.67:39437): tablet 18e3e3fff8094a2dbea19994119ac5ca (table test-table [id=1778cf78c2514a8392a32c54805cba8f]) successfully deleted
I20260806 03:54:06.065513 527 ts_tablet_manager.cc:1918] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:06.065656 527 log.cc:1196] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/18e3e3fff8094a2dbea19994119ac5ca
I20260806 03:54:06.065958 527 ts_tablet_manager.cc:1939] T 18e3e3fff8094a2dbea19994119ac5ca P 6ac4efcaabb5472c9c144b0d88d6d5d1: Deleting consensus metadata
I20260806 03:54:06.066418 32183 catalog_manager.cc:4841] TS 6ac4efcaabb5472c9c144b0d88d6d5d1 (127.12.15.66:40761): tablet 18e3e3fff8094a2dbea19994119ac5ca (table test-table [id=1778cf78c2514a8392a32c54805cba8f]) successfully deleted
W20260806 03:54:06.074620 32186 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:36485 (error 108)
I20260806 03:54:06.078272 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 32503
I20260806 03:54:06.139530 480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:06.139348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36485 (local address 127.0.0.1:53176)
0806 03:54:06.139411 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:06.139477 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:36485: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 03:54:06.139990 32186 catalog_manager.cc:4584] TS b1aa9e0452af4583aa7256b4b66a0959 (127.12.15.65:36485): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 18e3e3fff8094a2dbea19994119ac5ca: Network error: Client connection negotiation failed: client connection to 127.12.15.65:36485: connect: Connection refused (error 111)
I20260806 03:54:06.146349 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 32711
I20260806 03:54:06.174144 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 32154
I20260806 03:54:06.187270 12349 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260806 03:54:06.541851 12349 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260806 03:54:06.541952 12349 mini_ranger.cc:66] Stopping Ranger...
I20260806 03:54:06.886029 12349 mini_ranger.cc:68] Stopped Ranger
2026-08-06 03:54:06.886 UTC [30653] LOG: received smart shutdown request
2026-08-06 03:54:06.890 UTC [30653] LOG: background worker "logical replication launcher" (PID 30667) exited with exit code 1
2026-08-06 03:54:06.890 UTC [30659] LOG: shutting down
2026-08-06 03:54:07.040 UTC [30653] LOG: database system is shut down
2026-08-06T03:54:07Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (47597 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:54:07 dist-test-slave-bqcl krb5kdc[545](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:54:07 dist-test-slave-bqcl krb5kdc[545](info): set up 2 sockets
Aug 06 03:54:07 dist-test-slave-bqcl krb5kdc[545](info): commencing operation
krb5kdc: starting...
W20260806 03:54:08.266976 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:54:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:54:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:54:08.385351 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:44617
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:44617,127.12.15.125:34697,127.12.15.124:36049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:08.468497 563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:08.468701 563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:08.468722 563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:08.469868 563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:08.469911 563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:08.469924 563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:08.469949 563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:08.469971 563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:08.471554 563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:44617,127.12.15.125:34697,127.12.15.124:36049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:44617
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:08.471784 563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:08.472007 563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:08.476590 572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:08.476611 577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:08.476614 574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:08.476846 563 server_base.cc:1034] running on GCE node
I20260806 03:54:08.477106 563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:08.477393 563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:08.478540 563 hybrid_clock.cc:648] HybridClock initialized: now 1785988448478509 us; error 45 us; skew 500 ppm
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:08.481022 563 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:08.481457 563 webserver.cc:460] Webserver started at http://127.12.15.126:44229/ using document root <none> and password file <none>
I20260806 03:54:08.481676 563 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:08.481726 563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:08.481853 563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:08.482923 563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "626f79bc8cc3443da5418dbcef48468c"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "c22c6ca9f88f29a489da6b93ef01f008"
server_key_iv: "7d215beceeee9e6474bcfe10ee37928a"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.483265 563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "626f79bc8cc3443da5418dbcef48468c"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "c22c6ca9f88f29a489da6b93ef01f008"
server_key_iv: "7d215beceeee9e6474bcfe10ee37928a"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.484565 563 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:54:08.485311 586 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:08.485472 563 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:54:08.485572 563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "626f79bc8cc3443da5418dbcef48468c"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "c22c6ca9f88f29a489da6b93ef01f008"
server_key_iv: "7d215beceeee9e6474bcfe10ee37928a"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.485747 563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:08.495409 563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:08.497180 563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:08.497308 563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:08.501284 563 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:44617
I20260806 03:54:08.501332 669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:44617 every 8 connection(s)
I20260806 03:54:08.501641 563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:08.502506 670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:08.504060 670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:08.509907 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 563
I20260806 03:54:08.509991 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:54:08.510197 12349 external_mini_cluster.cc:1428] Setting key e8064683d2a5038ea3f041b9c52bda22
I20260806 03:54:08.512945 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.504717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44617 (local address 127.0.0.1:40422)
0806 03:54:08.504877 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.504914 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.505372 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.505637 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.505640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.505877 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.506043 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.506048 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.507108 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.507109 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.507783 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.507785 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.508200 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.508201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.508254 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.508635 (+ 381us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.508642 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.510005 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.511596 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.511603 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.511606 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.511789 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.511997 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.511998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.511998 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.512087 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.512402 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.512407 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.512467 (+ 60us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.512546 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.512768 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":103,"thread_start_us":67,"threads_started":1}
I20260806 03:54:08.512950 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.504749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40422 (local address 127.12.15.126:44617)
0806 03:54:08.504899 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.504907 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.504989 (+ 82us) server_negotiation.cc:407] Connection header received
0806 03:54:08.505492 (+ 503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.505494 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.505510 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.505589 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.506164 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.507015 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.507847 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.508143 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.508226 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.510108 (+ 1882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.510116 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.510116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.510144 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.511491 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.511860 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.511863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.511863 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.511940 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.512141 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.512141 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.512142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.512342 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.512413 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.512586 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.512712 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":117,"thread_start_us":73,"threads_started":1}
I20260806 03:54:08.513754 670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:54:08.514063 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.513954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:50948)
0806 03:54:08.513996 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.514040 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 03:54:08.514228 591 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
I20260806 03:54:08.514680 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.514590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:50960)
0806 03:54:08.514632 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.514659 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 03:54:08.514808 670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:34697: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.12.15.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab.
Entry for principal kudu/127.12.15.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab.
I20260806 03:54:08.535065 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.125:34697
--webserver_interface=127.12.15.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab
--principal=kudu/127.12.15.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:44617,127.12.15.125:34697,127.12.15.124:36049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:08.564014 670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } attempt: 1
I20260806 03:54:08.564474 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.564346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:50976)
0806 03:54:08.564404 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.564440 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:54:08.564872 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.564776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:50984)
0806 03:54:08.564822 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.564850 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260806 03:54:08.564962 670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:34697: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
W20260806 03:54:08.615942 679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:08.616156 679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:08.616186 679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:08.617305 679 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:08.617345 679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:08.617363 679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:08.617379 679 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:08.617393 679 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:08.618906 679 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:44617,127.12.15.125:34697,127.12.15.124:36049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab
--principal=kudu/127.12.15.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.125:34697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.15.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:08.619164 679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:08.619374 679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:08.624006 686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:08.623976 689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:08.624140 679 server_base.cc:1034] running on GCE node
W20260806 03:54:08.623996 687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:08.624477 679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:08.624750 679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:08.625887 679 hybrid_clock.cc:648] HybridClock initialized: now 1785988448625873 us; error 27 us; skew 500 ppm
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:08.628438 679 init.cc:377] Logged in from keytab as kudu/127.12.15.125@KRBTEST.COM (short username kudu)
I20260806 03:54:08.628854 679 webserver.cc:460] Webserver started at http://127.12.15.125:33695/ using document root <none> and password file <none>
I20260806 03:54:08.629030 679 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:08.629068 679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:08.629165 679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:08.630240 679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/data/instance:
uuid: "edf84dcb75a6466eb088f64ace759871"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "a336759ee41c9dcfb8ac0824e127204a"
server_key_iv: "fa463b51a718144537691059af04d16e"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.630592 679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/wal/instance:
uuid: "edf84dcb75a6466eb088f64ace759871"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "a336759ee41c9dcfb8ac0824e127204a"
server_key_iv: "fa463b51a718144537691059af04d16e"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.631811 679 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:08.632470 699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:08.632647 679 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:08.632732 679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/wal
uuid: "edf84dcb75a6466eb088f64ace759871"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "a336759ee41c9dcfb8ac0824e127204a"
server_key_iv: "fa463b51a718144537691059af04d16e"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.632795 679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:08.633106 670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } attempt: 2
I20260806 03:54:08.633482 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.633336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:50988)
0806 03:54:08.633398 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.633443 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 03:54:08.633816 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.633724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:50998)
0806 03:54:08.633763 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.633791 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 03:54:08.633898 670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:34697: Network error: Client connection negotiation failed: client connection to 127.12.15.125:34697: connect: Connection refused (error 111)
I20260806 03:54:08.651469 679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:08.653621 679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:08.653736 679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:08.657752 679 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.125:34697
I20260806 03:54:08.657804 771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.125:34697 every 8 connection(s)
I20260806 03:54:08.658247 679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
I20260806 03:54:08.659139 772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:08.659770 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 679
I20260806 03:54:08.659876 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-1/wal/instance
I20260806 03:54:08.660074 12349 external_mini_cluster.cc:1428] Setting key 891c5fb4ce36b7e59286220ecb0d0a60
I20260806 03:54:08.661096 772 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.124@KRBTEST.COM; defaulting to no policy
I20260806 03:54:08.670224 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.661879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40432 (local address 127.12.15.126:44617)
0806 03:54:08.661930 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.661932 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.662203 (+ 271us) server_negotiation.cc:407] Connection header received
0806 03:54:08.662669 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.662670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.662689 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.662766 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.663387 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.664086 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.665094 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.665430 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.665509 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.667545 (+ 2036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.667547 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.667547 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.667556 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.669001 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.669364 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.669365 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.669365 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.669418 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.669640 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.669641 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.669641 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.669755 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.669806 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.670048 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.670144 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:54:08.670408 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.661864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44617 (local address 127.0.0.1:40432)
0806 03:54:08.662049 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.662091 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.662578 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.662830 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.662833 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.663085 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.663290 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.663294 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.664187 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.664188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.665001 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.665004 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.665506 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.665506 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.665607 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.666079 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.666088 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.667451 (+ 1363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.669077 (+ 1626us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.669084 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.669086 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.669306 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.669465 (+ 159us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.669466 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.669466 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.669581 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.669811 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.669817 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.669940 (+ 123us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.670027 (+ 87us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.670187 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":95,"threads_started":1}
I20260806 03:54:08.671090 772 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.124@KRBTEST.COM" created.
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:54:08.678802 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.671351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:51010)
0806 03:54:08.671396 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.671405 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.671442 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.671722 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.671723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.671865 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.671923 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.671925 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.672757 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.672758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.673483 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.673485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.674096 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.674097 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.674182 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.674560 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.674565 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.675889 (+ 1324us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.677811 (+ 1922us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.677812 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.677812 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.677971 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.678224 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.678225 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.678225 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.678279 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.678580 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.678581 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.678615 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.678664 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.678738 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 03:54:08.678925 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.671407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51010 (local address 127.12.15.125:34697)
0806 03:54:08.671560 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.671565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.671588 (+ 23us) server_negotiation.cc:407] Connection header received
0806 03:54:08.671612 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.671614 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.671630 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.671707 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.671990 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.672675 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.673562 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.673996 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.674101 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.675979 (+ 1878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.675990 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.675990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.675998 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.677712 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.678054 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.678059 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.678059 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.678158 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.678331 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.678332 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.678332 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.678519 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.678601 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.678703 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.678817 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":17,"server-negotiator.queue_time_us":121,"thread_start_us":79,"threads_started":1}
I20260806 03:54:08.679608 772 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:54:08.679927 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.679824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60358)
0806 03:54:08.679864 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.679907 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 03:54:08.680076 703 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
I20260806 03:54:08.680459 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.680382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60362)
0806 03:54:08.680419 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.680443 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 03:54:08.680546 772 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:36049: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab.
Entry for principal kudu/127.12.15.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab.
I20260806 03:54:08.687139 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.124:36049
--webserver_interface=127.12.15.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab
--principal=kudu/127.12.15.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40911
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:44617,127.12.15.125:34697,127.12.15.124:36049
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:08.725080 670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } attempt: 3
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:54:08.729801 772 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } attempt: 1
I20260806 03:54:08.730229 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.730078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60366)
0806 03:54:08.730145 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.730191 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 03:54:08.730552 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.730458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60382)
0806 03:54:08.730505 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.730530 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260806 03:54:08.730633 772 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:36049: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
I20260806 03:54:08.732798 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.725520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:51016)
0806 03:54:08.725624 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.725639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.725705 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.725837 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.725838 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.726002 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.726071 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.726073 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.727022 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.727023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.727558 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.727560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.728030 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.728030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.728083 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.728554 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.728559 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.730048 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.731826 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.731827 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.731827 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.732062 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.732227 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.732228 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.732228 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.732294 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.732487 (+ 193us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.732488 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.732540 (+ 52us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.732606 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.732704 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260806 03:54:08.732805 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.725624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51016 (local address 127.12.15.125:34697)
0806 03:54:08.725696 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.725699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.725712 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:54:08.725748 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.725749 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.725762 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.725819 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.726166 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.726933 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.727637 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.727955 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.728037 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.730150 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.730151 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.730151 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.730158 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.731744 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.732128 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.732129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.732129 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.732179 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.732343 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.732344 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.732344 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.732440 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.732487 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.732663 (+ 176us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.732741 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260806 03:54:08.733253 670 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:54:08.733584 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.733479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60394)
0806 03:54:08.733521 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.733556 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 03:54:08.733984 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.733886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60402)
0806 03:54:08.733931 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.733959 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 03:54:08.734074 670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:36049: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
I20260806 03:54:08.765218 670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } attempt: 1
I20260806 03:54:08.765674 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.765551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60404)
0806 03:54:08.765607 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.765642 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 03:54:08.766057 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.765968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60414)
0806 03:54:08.766007 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.766035 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 03:54:08.766176 670 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:36049: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
W20260806 03:54:08.773941 782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:08.774168 782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:08.774202 782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:08.775336 782 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:08.775373 782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:08.775393 782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:08.775408 782 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:08.775425 782 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:08.776907 782 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40911
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:44617,127.12.15.125:34697,127.12.15.124:36049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab
--principal=kudu/127.12.15.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.124:36049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.15.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.782
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:08.777143 782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:08.777345 782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:08.781910 791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:08.781927 795 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:08.782027 782 server_base.cc:1034] running on GCE node
W20260806 03:54:08.782187 789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:08.782467 782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:08.782734 782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:08.783885 782 hybrid_clock.cc:648] HybridClock initialized: now 1785988448783870 us; error 28 us; skew 500 ppm
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:08.786432 782 init.cc:377] Logged in from keytab as kudu/127.12.15.124@KRBTEST.COM (short username kudu)
I20260806 03:54:08.786902 782 webserver.cc:460] Webserver started at http://127.12.15.124:41815/ using document root <none> and password file <none>
I20260806 03:54:08.787120 782 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:08.787170 782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:08.787307 782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:08.788465 782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/data/instance:
uuid: "7851f1ae9e1b417f9454b870784f3fd3"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "20c0ecfd51dc7fb7e9041684b6c7c62e"
server_key_iv: "ff0c31ed866e17bfb1e5103b7c3ac30e"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.788859 782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/wal/instance:
uuid: "7851f1ae9e1b417f9454b870784f3fd3"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "20c0ecfd51dc7fb7e9041684b6c7c62e"
server_key_iv: "ff0c31ed866e17bfb1e5103b7c3ac30e"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.790261 782 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:54:08.790977 803 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:08.791145 782 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:08.791241 782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/wal
uuid: "7851f1ae9e1b417f9454b870784f3fd3"
format_stamp: "Formatted at 2026-08-06 03:54:08 on dist-test-slave-bqcl"
server_key: "20c0ecfd51dc7fb7e9041684b6c7c62e"
server_key_iv: "ff0c31ed866e17bfb1e5103b7c3ac30e"
server_key_version: "encryptionkey@0"
I20260806 03:54:08.791309 782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:08.798760 772 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } attempt: 2
I20260806 03:54:08.799163 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.799020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60428)
0806 03:54:08.799077 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.799122 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 03:54:08.799487 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.799410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60442)
0806 03:54:08.799446 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.799470 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 03:54:08.799609 772 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:36049: Network error: Client connection negotiation failed: client connection to 127.12.15.124:36049: connect: Connection refused (error 111)
I20260806 03:54:08.811650 782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:08.813809 782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:08.813952 782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:08.817865 782 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.124:36049
I20260806 03:54:08.817904 881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.124:36049 every 8 connection(s)
I20260806 03:54:08.818280 782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
I20260806 03:54:08.819155 882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:08.820719 882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:54:08.822692 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 782
I20260806 03:54:08.822790 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACert.1785988295328500-12349-0/minicluster-data/master-2/wal/instance
I20260806 03:54:08.822994 12349 external_mini_cluster.cc:1428] Setting key 0aeac6d77bf6559dc32e3cae9cedec04
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
I20260806 03:54:08.829746 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.821409 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40434 (local address 127.12.15.126:44617)
0806 03:54:08.821465 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.821467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.821629 (+ 162us) server_negotiation.cc:407] Connection header received
0806 03:54:08.822068 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.822069 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.822078 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.822162 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.822795 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.823444 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.824470 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.824771 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.824843 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.826997 (+ 2154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.826998 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.826999 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.827006 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.828317 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.828740 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.828742 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.828742 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.828797 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.829054 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.829055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.829056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.829189 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.829257 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.829570 (+ 313us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.829649 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:54:08.829828 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.821355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:44617 (local address 127.0.0.1:40434)
0806 03:54:08.821510 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.821555 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.821990 (+ 435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.822249 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.822253 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.822497 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.822696 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.822700 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.823545 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.823546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.824390 (+ 844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.824392 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.824841 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.824841 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.824945 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.825426 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.825434 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.826912 (+ 1478us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.828394 (+ 1482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.828403 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.828405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.828675 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.828855 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.828856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.828857 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.828995 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.829260 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.829268 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.829431 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.829530 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.829649 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":16,"thread_start_us":65,"threads_started":1}
I20260806 03:54:08.830348 882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:54:08.831295 886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.823361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60450 (local address 127.12.15.124:36049)
0806 03:54:08.823594 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.823599 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.823622 (+ 23us) server_negotiation.cc:407] Connection header received
0806 03:54:08.823643 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.823645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.823664 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.823728 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.824285 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.825192 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.825876 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.826312 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.826388 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.828496 (+ 2108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.828503 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.828503 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.828516 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.830210 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.830518 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.830521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.830521 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.830581 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.830776 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.830777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.830777 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.830915 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.831059 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.831126 (+ 67us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.831197 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":203,"thread_start_us":90,"threads_started":1}
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:08.836262 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.830578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:34697 (local address 127.0.0.1:51024)
0806 03:54:08.830684 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.830694 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.830739 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.830874 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.830874 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.830997 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.831047 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.831049 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.831700 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.831701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.832266 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.832268 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.832597 (+ 329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.832598 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.832665 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.833009 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.833013 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.834015 (+ 1002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.835351 (+ 1336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.835352 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.835352 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.835499 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.835721 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.835723 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.835723 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.835786 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.836015 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.836016 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.836059 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.836125 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.836193 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260806 03:54:08.836321 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.830636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51024 (local address 127.12.15.125:34697)
0806 03:54:08.830691 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.830693 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.830727 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:54:08.830803 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.830804 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.830821 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.830864 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.831111 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.831632 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.832350 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.832540 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.832595 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.834090 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.834092 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.834092 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.834099 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.835285 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.835583 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.835583 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.835584 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.835642 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.835839 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.835840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.835841 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.835952 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.836004 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.836156 (+ 152us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.836229 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:54:08.836623 882 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:54:08.838600 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.831961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40438 (local address 127.12.15.126:44617)
0806 03:54:08.832004 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.832005 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.832022 (+ 17us) server_negotiation.cc:407] Connection header received
0806 03:54:08.832040 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.832041 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.832049 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.832093 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.832497 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.833023 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.833644 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.833859 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.833909 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.836100 (+ 2191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.836101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.836101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.836108 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.837539 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.837849 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.837850 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.837850 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.837901 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.838064 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.838064 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.838065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.838195 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.838397 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.838436 (+ 39us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.838506 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
I20260806 03:54:08.841323 670 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } attempt: 2
I20260806 03:54:08.842578 884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.837076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60458)
0806 03:54:08.837115 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.837124 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.837165 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.837274 (+ 109us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.837275 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.837390 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.837414 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.837415 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.837975 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.837976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.838473 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.838475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.838899 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.838900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.838945 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.839247 (+ 302us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.839251 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.840287 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.841629 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.841630 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.841631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.841802 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.841965 (+ 163us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.841965 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.841966 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.842033 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.842295 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.842296 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.842332 (+ 36us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.842393 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.842481 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 03:54:08.842701 886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.837116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60458 (local address 127.12.15.124:36049)
0806 03:54:08.837159 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.837161 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.837169 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:54:08.837216 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.837217 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.837224 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.837263 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.837470 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.837910 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.838605 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.838842 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.838900 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.840357 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.840358 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.840358 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.840364 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.841559 (+ 1195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.841860 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.841861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.841861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.841914 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.842093 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.842094 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.842094 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.842235 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.842294 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.842418 (+ 124us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.842534 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":49,"server-negotiator.queue_time_us":26}
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:54:08.844046 882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3: Bootstrap starting.
I20260806 03:54:08.844806 882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:08.845157 882 log.cc:824] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:08.846196 882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3: No bootstrap required, opened a new log
I20260806 03:54:08.846532 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.839141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51026 (local address 127.12.15.125:34697)
0806 03:54:08.839177 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.839179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.839301 (+ 122us) server_negotiation.cc:407] Connection header received
0806 03:54:08.839342 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.839342 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.839352 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.839412 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.839823 (+ 411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.840304 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.841065 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.841282 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.841314 (+ 32us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.843748 (+ 2434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.843749 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.843750 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.843757 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.845301 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.845697 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.845698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.845698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.845758 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.845964 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.845965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.845965 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.846103 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.846326 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.846375 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.846446 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:54:08.847954 882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.848099 882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:08.848145 882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7851f1ae9e1b417f9454b870784f3fd3, State: Initialized, Role: FOLLOWER
I20260806 03:54:08.848265 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:54:08.848263 882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.848515 890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.848599 890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:54:08.848611 882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
W20260806 03:54:08.851737 916 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:54:08.851796 916 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:54:08.853266 672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.841555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60470)
0806 03:54:08.841609 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.841620 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.841661 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.841904 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.841905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.842051 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.842087 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.842088 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.843842 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.843843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.844476 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.844479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.845673 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.845674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.845735 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.848686 (+ 2951us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.848690 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.850088 (+ 1398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.852092 (+ 2004us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.852093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.852093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.852310 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.852558 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.852559 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.852559 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.852627 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.852950 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.852951 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.852999 (+ 48us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.853065 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.853164 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 03:54:08.853658 887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.841636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60470 (local address 127.12.15.124:36049)
0806 03:54:08.841789 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.841790 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.841798 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:54:08.841816 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.841817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.841826 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.841897 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.842198 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.843770 (+ 1572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.844558 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.845596 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.845655 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.850256 (+ 4601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.850257 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.850258 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.850265 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.851999 (+ 1734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.852393 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.852395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.852395 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.852462 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.852699 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.852699 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.852700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.852852 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.852918 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.853459 (+ 541us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.853559 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":88,"threads_started":1}
I20260806 03:54:08.852415 782 master_runner.cc:186] Error getting master registration for 127.12.15.126:44617: OK, instance_id { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" instance_seqno: 1785988448500171 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:54:08.854818 782 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:54:08.855599 670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c: Bootstrap starting.
I20260806 03:54:08.856338 670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:08.856676 670 log.cc:824] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:08.857517 670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c: No bootstrap required, opened a new log
I20260806 03:54:08.859320 777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.849585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51032 (local address 127.12.15.125:34697)
0806 03:54:08.849625 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.849626 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.849727 (+ 101us) server_negotiation.cc:407] Connection header received
0806 03:54:08.849889 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.849890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.849899 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.849927 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.851499 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.852118 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.853069 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.853356 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.853399 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.855552 (+ 2153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.855554 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.855554 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.855561 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.857590 (+ 2029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.858251 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.858253 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.858253 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.858310 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.858508 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.858509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.858510 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.858631 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.858727 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.858919 (+ 192us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.858996 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:54:08.859625 670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.859772 670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:08.859812 670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 626f79bc8cc3443da5418dbcef48468c, State: Initialized, Role: FOLLOWER
I20260806 03:54:08.859923 670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.860242 670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:08.860380 673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.849703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.12.15.126:44617)
0806 03:54:08.849748 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.849749 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.850260 (+ 511us) server_negotiation.cc:407] Connection header received
0806 03:54:08.850353 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.850353 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.850363 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.850389 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.851060 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.851701 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.853661 (+ 1960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.853945 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.853989 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.856679 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.856681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.856681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.856688 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.858402 (+ 1714us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.858753 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.858754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.858754 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.858810 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.859115 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.859116 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.859116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.859244 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.859313 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.860222 (+ 909us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.860300 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:54:08.860767 920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.860920 920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:54:08.862169 935 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:54:08.862227 935 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:54:08.863267 563 master_runner.cc:186] Error getting master registration for 127.12.15.125:34697: OK, instance_id { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" instance_seqno: 1785988448656651 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:54:08.865507 563 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:54:08.866755 886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.850006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.12.15.124:36049)
0806 03:54:08.850601 (+ 595us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.850603 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.852869 (+ 2266us) server_negotiation.cc:407] Connection header received
0806 03:54:08.852947 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.852948 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.852957 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.852983 (+ 26us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.854350 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.854994 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.856964 (+ 1970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.857234 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.857277 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.861178 (+ 3901us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.861179 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.861179 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.861186 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.862690 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.865598 (+ 2908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.865600 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.865600 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.865662 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.865889 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.865890 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.865890 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.866016 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.866098 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.866573 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.866654 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576}
I20260806 03:54:08.890746 772 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } attempt: 3
Aug 06 03:54:08 dist-test-slave-bqcl krb5kdc[545](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988448, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
I20260806 03:54:08.899199 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.891028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:36049 (local address 127.0.0.1:60482)
0806 03:54:08.891079 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:08.891090 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:08.891126 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:08.891309 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:08.891310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:08.891503 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:08.891558 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.891560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.892469 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.892470 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.893302 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:08.893305 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.893791 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.893792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:08.893860 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.894386 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:08.894392 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:08.895961 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:08.897789 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.897790 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.897791 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.897999 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.898514 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:08.898515 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:08.898515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:08.898598 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:08.898861 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:08.898862 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:08.898912 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 03:54:08.898984 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:54:08.899091 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:54:08.899201 886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:08.891096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60482 (local address 127.12.15.124:36049)
0806 03:54:08.891195 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:08.891197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:08.891204 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:54:08.891225 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:08.891226 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:08.891235 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:08.891287 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:08.891635 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.892386 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.893381 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:08.893721 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:08.893790 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:08.896064 (+ 2274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:08.896066 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:08.896066 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:08.896074 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:08.897695 (+ 1621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.898382 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.898383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.898383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.898448 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:08.898662 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:08.898663 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:08.898663 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:08.898798 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:08.898866 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:08.899015 (+ 149us) server_negotiation.cc:299] Negotiation successful
0806 03:54:08.899092 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260806 03:54:08.901259 772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871: Bootstrap starting.
I20260806 03:54:08.902050 772 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:08.902408 772 log.cc:824] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:08.903308 772 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871: No bootstrap required, opened a new log
I20260806 03:54:08.905332 772 raft_consensus.cc:348] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.905481 772 raft_consensus.cc:374] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:08.905515 772 raft_consensus.cc:729] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edf84dcb75a6466eb088f64ace759871, State: Initialized, Role: FOLLOWER
I20260806 03:54:08.905929 772 consensus_queue.cc:260] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.906322 772 sys_catalog.cc:564] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:08.906437 938 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.906574 938 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:54:08.908214 957 catalog_manager.cc:1558] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:54:08.908321 957 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:54:08.910226 679 master_runner.cc:263] No leader master found from master edf84dcb75a6466eb088f64ace759871
I20260806 03:54:08.910280 679 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:54:08.916056 920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:08.916169 920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.916390 920 leader_election.cc:288] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edf84dcb75a6466eb088f64ace759871 (127.12.15.125:34697), 7851f1ae9e1b417f9454b870784f3fd3 (127.12.15.124:36049)
I20260806 03:54:08.916713 736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "626f79bc8cc3443da5418dbcef48468c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf84dcb75a6466eb088f64ace759871" is_pre_election: true
I20260806 03:54:08.916858 736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 626f79bc8cc3443da5418dbcef48468c in term 0.
I20260806 03:54:08.917023 591 leader_election.cc:302] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 626f79bc8cc3443da5418dbcef48468c, edf84dcb75a6466eb088f64ace759871; no voters:
I20260806 03:54:08.917330 849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "626f79bc8cc3443da5418dbcef48468c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7851f1ae9e1b417f9454b870784f3fd3" is_pre_election: true
I20260806 03:54:08.917464 849 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 626f79bc8cc3443da5418dbcef48468c in term 0.
I20260806 03:54:08.917872 920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:54:08.917934 920 raft_consensus.cc:482] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:54:08.917963 920 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:08.918831 920 raft_consensus.cc:504] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.918954 920 leader_election.cc:288] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [CANDIDATE]: Term 1 election: Requested vote from peers edf84dcb75a6466eb088f64ace759871 (127.12.15.125:34697), 7851f1ae9e1b417f9454b870784f3fd3 (127.12.15.124:36049)
I20260806 03:54:08.919158 736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "626f79bc8cc3443da5418dbcef48468c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edf84dcb75a6466eb088f64ace759871"
I20260806 03:54:08.919241 736 raft_consensus.cc:3037] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:08.920043 736 raft_consensus.cc:2454] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 626f79bc8cc3443da5418dbcef48468c in term 1.
I20260806 03:54:08.920212 591 leader_election.cc:302] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 626f79bc8cc3443da5418dbcef48468c, edf84dcb75a6466eb088f64ace759871; no voters:
I20260806 03:54:08.920389 849 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "626f79bc8cc3443da5418dbcef48468c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7851f1ae9e1b417f9454b870784f3fd3"
I20260806 03:54:08.920463 849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:08.921186 849 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 626f79bc8cc3443da5418dbcef48468c in term 1.
I20260806 03:54:08.921695 920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:08.921859 920 raft_consensus.cc:686] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [term 1 LEADER]: Becoming Leader. State: Replica: 626f79bc8cc3443da5418dbcef48468c, State: Running, Role: LEADER
I20260806 03:54:08.921945 920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } }
I20260806 03:54:08.922348 920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 626f79bc8cc3443da5418dbcef48468c. Latest consensus state: current_term: 1 leader_uuid: "626f79bc8cc3443da5418dbcef48468c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.922418 920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:08.922573 964 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:08.922725 964 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:54:08.924674 849 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [term 1 FOLLOWER]: Refusing update from remote peer 626f79bc8cc3443da5418dbcef48468c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:08.924906 736 raft_consensus.cc:1261] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [term 1 FOLLOWER]: Refusing update from remote peer 626f79bc8cc3443da5418dbcef48468c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:08.925307 920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:08.925449 920 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [LEADER]: Connected to new peer: Peer: permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:08.926620 938 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 626f79bc8cc3443da5418dbcef48468c. Latest consensus state: current_term: 1 leader_uuid: "626f79bc8cc3443da5418dbcef48468c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.926704 938 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:54:08.927630 890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 626f79bc8cc3443da5418dbcef48468c. Latest consensus state: current_term: 1 leader_uuid: "626f79bc8cc3443da5418dbcef48468c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.927719 890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:54:08.928773 920 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "626f79bc8cc3443da5418dbcef48468c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.928855 920 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:08.929169 890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "626f79bc8cc3443da5418dbcef48468c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.929263 890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:54:08.929453 965 mvcc.cc:205] Tried to move back new op lower bound from 7315408686792675328 to 7315408686784667648. Current Snapshot: MvccSnapshot[applied={T|T < 7315408686792675328}]
I20260806 03:54:08.929932 969 mvcc.cc:205] Tried to move back new op lower bound from 7315408686792675328 to 7315408686784667648. Current Snapshot: MvccSnapshot[applied={T|T < 7315408686792675328}]
I20260806 03:54:08.930462 964 catalog_manager.cc:1347] Generated new cluster ID: 1bcb03628cf3442889068cced3ccac42
I20260806 03:54:08.930516 964 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:54:08.936266 960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "626f79bc8cc3443da5418dbcef48468c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.936389 960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:08.936918 938 sys_catalog.cc:455] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "626f79bc8cc3443da5418dbcef48468c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "626f79bc8cc3443da5418dbcef48468c" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 44617 } } peers { permanent_uuid: "edf84dcb75a6466eb088f64ace759871" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 34697 } } peers { permanent_uuid: "7851f1ae9e1b417f9454b870784f3fd3" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 36049 } } }
I20260806 03:54:08.937002 938 sys_catalog.cc:458] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:54:08.937186 966 mvcc.cc:205] Tried to move back new op lower bound from 7315408686792675328 to 7315408686784667648. Current Snapshot: MvccSnapshot[applied={T|T < 7315408686792675328}]
I20260806 03:54:08.947317 964 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:54:08.948078 964 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:54:08.952929 964 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 626f79bc8cc3443da5418dbcef48468c: Generated new TSK 0
I20260806 03:54:08.953110 964 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:54:09.852240 916 catalog_manager.cc:1259] Loaded cluster ID: 1bcb03628cf3442889068cced3ccac42
I20260806 03:54:09.852319 916 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3: loading cluster ID for follower catalog manager: success
I20260806 03:54:09.853132 916 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3: acquiring CA information for follower catalog manager: success
I20260806 03:54:09.853436 916 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7851f1ae9e1b417f9454b870784f3fd3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:54:09.908821 957 catalog_manager.cc:1259] Loaded cluster ID: 1bcb03628cf3442889068cced3ccac42
I20260806 03:54:09.908902 957 catalog_manager.cc:1552] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871: loading cluster ID for follower catalog manager: success
I20260806 03:54:09.909737 957 catalog_manager.cc:1574] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871: acquiring CA information for follower catalog manager: success
I20260806 03:54:09.910049 957 catalog_manager.cc:1602] T 00000000000000000000000000000000 P edf84dcb75a6466eb088f64ace759871: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:54:10.008862 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 563
I20260806 03:54:10.020094 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 679
I20260806 03:54:10.029825 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 782
2026-08-06T03:54:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2898 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:54:10 dist-test-slave-bqcl krb5kdc[989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:54:10 dist-test-slave-bqcl krb5kdc[989](info): set up 2 sockets
Aug 06 03:54:10 dist-test-slave-bqcl krb5kdc[989](info): commencing operation
krb5kdc: starting...
W20260806 03:54:11.168761 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.116s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:54:11 dist-test-slave-bqcl krb5kdc[989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:54:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:54:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:54:11.292865 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:34659
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:37997
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:34659
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:11.377938 1007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:11.378216 1007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:11.378255 1007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:11.379423 1007 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:11.379480 1007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:11.379504 1007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:11.379526 1007 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:11.379552 1007 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:11.381081 1007 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:37997
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:34659
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:34659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:11.381345 1007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:11.381595 1007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:11.386750 1017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:11.386729 1015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:11.386839 1014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:11.386885 1007 server_base.cc:1034] running on GCE node
I20260806 03:54:11.387238 1007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:11.387542 1007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:11.388716 1007 hybrid_clock.cc:648] HybridClock initialized: now 1785988451388691 us; error 37 us; skew 500 ppm
Aug 06 03:54:11 dist-test-slave-bqcl krb5kdc[989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988451, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:11.391484 1007 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:11.391921 1007 webserver.cc:460] Webserver started at http://127.12.15.126:36875/ using document root <none> and password file <none>
I20260806 03:54:11.392127 1007 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:11.392177 1007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:11.392323 1007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:11.393397 1007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "ad63061968a94cb69ed89a2082d4c11e"
format_stamp: "Formatted at 2026-08-06 03:54:11 on dist-test-slave-bqcl"
server_key: "aa57c92f2797582b5738f50368895d12"
server_key_iv: "fe2c79cca067a040e22cb7da5278c9e8"
server_key_version: "encryptionkey@0"
I20260806 03:54:11.393736 1007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "ad63061968a94cb69ed89a2082d4c11e"
format_stamp: "Formatted at 2026-08-06 03:54:11 on dist-test-slave-bqcl"
server_key: "aa57c92f2797582b5738f50368895d12"
server_key_iv: "fe2c79cca067a040e22cb7da5278c9e8"
server_key_version: "encryptionkey@0"
I20260806 03:54:11.395288 1007 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:54:11.396075 1024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:11.396260 1007 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:11.396353 1007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "ad63061968a94cb69ed89a2082d4c11e"
format_stamp: "Formatted at 2026-08-06 03:54:11 on dist-test-slave-bqcl"
server_key: "aa57c92f2797582b5738f50368895d12"
server_key_iv: "fe2c79cca067a040e22cb7da5278c9e8"
server_key_version: "encryptionkey@0"
I20260806 03:54:11.396414 1007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:11.412729 1007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:11.414649 1007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:11.414795 1007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:11.418810 1007 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:34659
I20260806 03:54:11.418851 1099 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:34659 every 8 connection(s)
I20260806 03:54:11.419195 1007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:11.420076 1100 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:11.422452 1100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e: Bootstrap starting.
I20260806 03:54:11.423033 1100 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:11.423314 1100 log.cc:824] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:11.423959 1100 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e: No bootstrap required, opened a new log
I20260806 03:54:11.425529 1100 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad63061968a94cb69ed89a2082d4c11e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34659 } }
I20260806 03:54:11.425671 1100 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:11.425707 1100 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad63061968a94cb69ed89a2082d4c11e, State: Initialized, Role: FOLLOWER
I20260806 03:54:11.425848 1100 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad63061968a94cb69ed89a2082d4c11e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34659 } }
I20260806 03:54:11.425908 1100 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:54:11.425930 1100 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:54:11.425976 1100 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:11.426502 1100 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad63061968a94cb69ed89a2082d4c11e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34659 } }
I20260806 03:54:11.426596 1100 leader_election.cc:302] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad63061968a94cb69ed89a2082d4c11e; no voters:
I20260806 03:54:11.426757 1100 leader_election.cc:288] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:54:11.426786 1104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:11.426923 1104 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [term 1 LEADER]: Becoming Leader. State: Replica: ad63061968a94cb69ed89a2082d4c11e, State: Running, Role: LEADER
I20260806 03:54:11.426994 1100 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:11.427026 1104 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad63061968a94cb69ed89a2082d4c11e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34659 } }
I20260806 03:54:11.427378 1106 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad63061968a94cb69ed89a2082d4c11e. Latest consensus state: current_term: 1 leader_uuid: "ad63061968a94cb69ed89a2082d4c11e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad63061968a94cb69ed89a2082d4c11e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34659 } } }
I20260806 03:54:11.427503 1105 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad63061968a94cb69ed89a2082d4c11e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad63061968a94cb69ed89a2082d4c11e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 34659 } } }
I20260806 03:54:11.427520 1106 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:11.427592 1105 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:11.427991 1116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:11.428406 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 1007
I20260806 03:54:11.428515 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:54:11.428529 1116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:54:11.428730 12349 external_mini_cluster.cc:1428] Setting key 807de3050dbd72017d12df2942a37738
I20260806 03:54:11.430068 1116 catalog_manager.cc:1347] Generated new cluster ID: 635847c9dba64213ba2ebd3acae52f04
I20260806 03:54:11.430119 1116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:54:11 dist-test-slave-bqcl krb5kdc[989](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:11.438199 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:11.429372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42030 (local address 127.12.15.126:34659)
0806 03:54:11.429572 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:11.429582 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:11.429793 (+ 211us) server_negotiation.cc:407] Connection header received
0806 03:54:11.429943 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:11.429950 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:11.430005 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:11.430160 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:11.430779 (+ 619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:11.431699 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:11.432430 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:11.432866 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:11.432962 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:11.435183 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:11.435187 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:11.435188 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:11.435200 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:11.436724 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:11.437046 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:11.437049 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:11.437050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:11.437151 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:11.437328 (+ 177us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:11.437329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:11.437329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:11.437560 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:11.437731 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:11.437833 (+ 102us) server_negotiation.cc:299] Negotiation successful
0806 03:54:11.437969 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":92,"threads_started":1}
I20260806 03:54:11.439009 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:54:11.449668 1116 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:54:11.450372 1116 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:54:11.462918 1116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad63061968a94cb69ed89a2082d4c11e: Generated new TSK 0
I20260806 03:54:11.463132 1116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:54:11.492654 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 1007
2026-08-06T03:54:11Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1466 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:54:11 dist-test-slave-bqcl krb5kdc[1141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:54:11 dist-test-slave-bqcl krb5kdc[1141](info): set up 2 sockets
Aug 06 03:54:11 dist-test-slave-bqcl krb5kdc[1141](info): commencing operation
krb5kdc: starting...
W20260806 03:54:12.636305 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.117s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:54:12 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:54:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:54:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:54:12.760399 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38993
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:38993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:12.842173 1163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:12.842391 1163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:12.842410 1163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:12.843495 1163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:12.843529 1163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:12.843541 1163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:12.843552 1163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:12.843568 1163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:12.844974 1163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:38993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:12.845158 1163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:12.845363 1163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:12.849996 1175 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:12.850050 1170 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:12.850144 1163 server_base.cc:1034] running on GCE node
W20260806 03:54:12.850073 1171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:12.850474 1163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:12.850747 1163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:12.851895 1163 hybrid_clock.cc:648] HybridClock initialized: now 1785988452851881 us; error 31 us; skew 500 ppm
Aug 06 03:54:12 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988452, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:12.854399 1163 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:12.854818 1163 webserver.cc:460] Webserver started at http://127.12.15.126:39891/ using document root <none> and password file <none>
I20260806 03:54:12.855064 1163 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:12.855118 1163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:12.855254 1163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:12.856436 1163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "0e320a4526f24e91845b15e1f8e42f1a"
format_stamp: "Formatted at 2026-08-06 03:54:12 on dist-test-slave-bqcl"
server_key: "4d17d81d4dcfc23850b7359e4843d663"
server_key_iv: "08c1a32342fca81f6361ee64c0c60f36"
server_key_version: "encryptionkey@0"
I20260806 03:54:12.856809 1163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "0e320a4526f24e91845b15e1f8e42f1a"
format_stamp: "Formatted at 2026-08-06 03:54:12 on dist-test-slave-bqcl"
server_key: "4d17d81d4dcfc23850b7359e4843d663"
server_key_iv: "08c1a32342fca81f6361ee64c0c60f36"
server_key_version: "encryptionkey@0"
I20260806 03:54:12.858052 1163 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:54:12.858768 1187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:12.858945 1163 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:12.859032 1163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "0e320a4526f24e91845b15e1f8e42f1a"
format_stamp: "Formatted at 2026-08-06 03:54:12 on dist-test-slave-bqcl"
server_key: "4d17d81d4dcfc23850b7359e4843d663"
server_key_iv: "08c1a32342fca81f6361ee64c0c60f36"
server_key_version: "encryptionkey@0"
I20260806 03:54:12.859097 1163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:12.872115 1163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:12.874398 1163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:12.874542 1163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:12.878454 1163 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:38993
I20260806 03:54:12.878489 1270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:38993 every 8 connection(s)
I20260806 03:54:12.878829 1163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:12.879817 1271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:12.882211 1271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Bootstrap starting.
I20260806 03:54:12.882767 1271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:12.883030 1271 log.cc:824] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:12.883677 1271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: No bootstrap required, opened a new log
I20260806 03:54:12.885252 1271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:12.885392 1271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:12.885414 1271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e320a4526f24e91845b15e1f8e42f1a, State: Initialized, Role: FOLLOWER
I20260806 03:54:12.885483 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 1163
I20260806 03:54:12.885542 1271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:12.885592 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:54:12.885627 1271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:54:12.885667 1271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:54:12.885717 1271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:12.885805 12349 external_mini_cluster.cc:1428] Setting key 673df23767e5e8127a9d1fb46269fc49
I20260806 03:54:12.886669 1271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:12.886787 1271 leader_election.cc:302] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e320a4526f24e91845b15e1f8e42f1a; no voters:
I20260806 03:54:12.886961 1271 leader_election.cc:288] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:54:12.886996 1277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:12.887137 1277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 1 LEADER]: Becoming Leader. State: Replica: 0e320a4526f24e91845b15e1f8e42f1a, State: Running, Role: LEADER
I20260806 03:54:12.887234 1271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:12.887265 1277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:12.887634 1278 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e320a4526f24e91845b15e1f8e42f1a. Latest consensus state: current_term: 1 leader_uuid: "0e320a4526f24e91845b15e1f8e42f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } } }
I20260806 03:54:12.887797 1278 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:12.888226 1289 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:12.888729 1289 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:54:12.888705 1280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0e320a4526f24e91845b15e1f8e42f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } } }
I20260806 03:54:12.888953 1280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:12.890362 1289 catalog_manager.cc:1347] Generated new cluster ID: dfb9522ffd75459cb200af9dfc58c67c
I20260806 03:54:12.890412 1289 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:54:12 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988452, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:12.897707 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:12.886291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50684 (local address 127.12.15.126:38993)
0806 03:54:12.886452 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:12.886460 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:12.886486 (+ 26us) server_negotiation.cc:407] Connection header received
0806 03:54:12.886582 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:12.886587 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:12.886627 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:12.886749 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:12.887547 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:12.888953 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:12.890029 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:12.890654 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:12.890771 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:12.893435 (+ 2664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:12.893439 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:12.893439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:12.893454 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:12.896092 (+ 2638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:12.896474 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:12.896477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:12.896477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:12.896605 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:12.896791 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:12.896792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:12.896792 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:12.897105 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:12.897284 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:12.897352 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 03:54:12.897492 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":74,"threads_started":1}
I20260806 03:54:12.904428 1289 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:54:12.905131 1289 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:54:12.916285 1289 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Generated new TSK 0
I20260806 03:54:12.916497 1289 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:54:12.946916 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38993
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:13.029237 1311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:13.029433 1311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:13.029454 1311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:13.030543 1311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:13.030577 1311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:13.030601 1311 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:54:13.032016 1311 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:13.032197 1311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:13.032372 1311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:13.037118 1320 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.037122 1323 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.037197 1319 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.037279 1311 server_base.cc:1034] running on GCE node
I20260806 03:54:13.037606 1311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:13.037843 1311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:13.038995 1311 hybrid_clock.cc:648] HybridClock initialized: now 1785988453038976 us; error 32 us; skew 500 ppm
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:13.041441 1311 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:54:13.041910 1311 webserver.cc:460] Webserver started at http://127.12.15.65:33529/ using document root <none> and password file <none>
I20260806 03:54:13.042111 1311 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:13.042202 1311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:13.042341 1311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:13.043351 1311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "5268740ff3d342a9a1dc1d7a9dc63c82"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "b84bbe50a75ee2df7e4c7c0a8b7983da"
server_key_iv: "40789113fa0f328a81943f8151e765b1"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.043710 1311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "5268740ff3d342a9a1dc1d7a9dc63c82"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "b84bbe50a75ee2df7e4c7c0a8b7983da"
server_key_iv: "40789113fa0f328a81943f8151e765b1"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.045035 1311 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:54:13.045744 1333 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.045908 1311 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:54:13.045990 1311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "5268740ff3d342a9a1dc1d7a9dc63c82"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "b84bbe50a75ee2df7e4c7c0a8b7983da"
server_key_iv: "40789113fa0f328a81943f8151e765b1"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.046043 1311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:13.063642 1311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:13.065745 1311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:13.065860 1311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:13.066056 1311 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:13.066390 1311 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:13.066424 1311 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.066447 1311 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:13.066461 1311 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.072180 1311 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:35647
I20260806 03:54:13.072227 1504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:35647 every 8 connection(s)
I20260806 03:54:13.072541 1311 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:13.081429 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.073282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:35305 (local address 127.12.15.126:38993)
0806 03:54:13.073346 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.073348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.073533 (+ 185us) server_negotiation.cc:407] Connection header received
0806 03:54:13.074083 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.074083 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.074098 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.074168 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:13.075118 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.075620 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.076558 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.076755 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.076797 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.078904 (+ 2107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:13.078905 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:13.078906 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:13.078913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:13.080124 (+ 1211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.080518 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.080519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.080520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.080575 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.080798 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.080799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.080799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.080914 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:13.080969 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.081233 (+ 264us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.081331 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:54:13.081604 1507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.073238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:35305)
0806 03:54:13.073414 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.073454 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.073993 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.074244 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.074248 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.074829 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:13.075029 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.075035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.075681 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.075681 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.076497 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.076500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.076817 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.076818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.076932 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.077426 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:13.077435 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:13.078808 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:13.080213 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.080221 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.080223 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.080446 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.080630 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.080630 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.080631 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.080741 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.080967 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:13.080974 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:13.081112 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.081209 (+ 97us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.081375 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":88,"threads_started":1}
I20260806 03:54:13.082096 1505 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38993
I20260806 03:54:13.082230 1505 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:13.082352 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 1311
I20260806 03:54:13.082427 1505 heartbeater.cc:511] Master 127.12.15.126:38993 requested a full tablet report, sending...
I20260806 03:54:13.082450 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:54:13.082633 12349 external_mini_cluster.cc:1428] Setting key 9261947a8d74c8f554665620a153a9f0
I20260806 03:54:13.083035 1217 ts_manager.cc:194] Registered new tserver with Master: 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647)
I20260806 03:54:13.083765 1217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:35305
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:54:13.107175 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38993
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:13.189105 1510 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:13.189322 1510 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:13.189353 1510 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:13.190496 1510 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:13.190538 1510 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:13.190573 1510 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:54:13.192031 1510 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1510
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:13.192272 1510 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:13.192488 1510 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:13.197373 1519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.197490 1510 server_base.cc:1034] running on GCE node
W20260806 03:54:13.197378 1523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.197373 1518 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.197777 1510 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:13.198036 1510 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:13.199177 1510 hybrid_clock.cc:648] HybridClock initialized: now 1785988453199178 us; error 42 us; skew 500 ppm
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:13.201640 1510 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:54:13.202088 1510 webserver.cc:460] Webserver started at http://127.12.15.66:39791/ using document root <none> and password file <none>
I20260806 03:54:13.202342 1510 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:13.202391 1510 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:13.202520 1510 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:13.203539 1510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "37cc06ea59e37912ff4925cb4b8e9fad"
server_key_iv: "49c1e0a0cce28d3f746aca5fe49e96e1"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.203868 1510 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "37cc06ea59e37912ff4925cb4b8e9fad"
server_key_iv: "49c1e0a0cce28d3f746aca5fe49e96e1"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.205206 1510 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:54:13.205869 1537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.206033 1510 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:54:13.206120 1510 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "37cc06ea59e37912ff4925cb4b8e9fad"
server_key_iv: "49c1e0a0cce28d3f746aca5fe49e96e1"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.206223 1510 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:13.232055 1510 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:13.233994 1510 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:13.234150 1510 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:13.234377 1510 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:13.234685 1510 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:13.234722 1510 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.234752 1510 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:13.234776 1510 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.240233 1510 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:41783
I20260806 03:54:13.240280 1717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:41783 every 8 connection(s)
I20260806 03:54:13.240582 1510 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:54:13.242516 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 1510
I20260806 03:54:13.242609 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:54:13.242807 12349 external_mini_cluster.cc:1428] Setting key 1de62cc073c95338d5630fe161a4b587
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:13.249852 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.241273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:38305 (local address 127.12.15.126:38993)
0806 03:54:13.241325 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.241327 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.241509 (+ 182us) server_negotiation.cc:407] Connection header received
0806 03:54:13.241979 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.241980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.241989 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.242042 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:13.243106 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.243783 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.244764 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.245037 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.245110 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.247360 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:13.247362 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:13.247362 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:13.247370 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:13.248585 (+ 1215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.248943 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.248944 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.248945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.249004 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.249220 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.249221 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.249222 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.249340 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:13.249397 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.249656 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.249760 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:54:13.250075 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.241235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:38305)
0806 03:54:13.241393 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.241435 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.241907 (+ 472us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.242100 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.242103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.242756 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:13.243015 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.243023 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.243880 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.243881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.244675 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.244678 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.245125 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.245126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.245292 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.245757 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:13.245768 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:13.247274 (+ 1506us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:13.248649 (+ 1375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.248657 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.248659 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.248875 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.249057 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.249058 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.249058 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.249163 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.249394 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:13.249401 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:13.249539 (+ 138us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.249628 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.249858 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260806 03:54:13.250542 1719 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38993
I20260806 03:54:13.250633 1719 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:13.250821 1719 heartbeater.cc:511] Master 127.12.15.126:38993 requested a full tablet report, sending...
I20260806 03:54:13.251181 1217 ts_manager.cc:194] Registered new tserver with Master: e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783)
I20260806 03:54:13.251540 1217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:38305
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:54:13.269008 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38993
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:13.352440 1727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:13.352646 1727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:13.352669 1727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:13.353732 1727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:13.353765 1727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:13.353789 1727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:54:13.355211 1727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:13.355419 1727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:13.355595 1727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:13.360500 1735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.360499 1733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.360477 1732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.360558 1727 server_base.cc:1034] running on GCE node
I20260806 03:54:13.360908 1727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:13.361140 1727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:13.362286 1727 hybrid_clock.cc:648] HybridClock initialized: now 1785988453362263 us; error 36 us; skew 500 ppm
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:13.364727 1727 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:54:13.365198 1727 webserver.cc:460] Webserver started at http://127.12.15.67:42835/ using document root <none> and password file <none>
I20260806 03:54:13.365434 1727 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:13.365482 1727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:13.365607 1727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:13.366690 1727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "516e1c592f424ebd8ec8487398ef7cc1"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "6aae94d1091a762db0f14321f1c0771a"
server_key_iv: "dd42fd4723ba93965157ad3050b6279c"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.367034 1727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "516e1c592f424ebd8ec8487398ef7cc1"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "6aae94d1091a762db0f14321f1c0771a"
server_key_iv: "dd42fd4723ba93965157ad3050b6279c"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.368269 1727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:54:13.368963 1747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.369122 1727 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:13.369207 1727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "516e1c592f424ebd8ec8487398ef7cc1"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "6aae94d1091a762db0f14321f1c0771a"
server_key_iv: "dd42fd4723ba93965157ad3050b6279c"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.369280 1727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:13.389657 1727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:13.391877 1727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:13.392017 1727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:13.392241 1727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:13.392547 1727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:13.392581 1727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.392611 1727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:13.392637 1727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.398414 1727 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:37347
I20260806 03:54:13.398478 1919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:37347 every 8 connection(s)
I20260806 03:54:13.398769 1727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:13.405632 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 1727
I20260806 03:54:13.405754 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:54:13.405948 12349 external_mini_cluster.cc:1428] Setting key 4084befb23305c079adb690bdbea5d30
I20260806 03:54:13.407729 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.399431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:45029 (local address 127.12.15.126:38993)
0806 03:54:13.399513 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.399516 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.399700 (+ 184us) server_negotiation.cc:407] Connection header received
0806 03:54:13.400223 (+ 523us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.400224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.400234 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.400274 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:13.401174 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.401665 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.402698 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.402879 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.402928 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.405017 (+ 2089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:13.405019 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:13.405019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:13.405025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:13.406521 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.406891 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.406892 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.406893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.406957 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.407160 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.407161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.407161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.407256 (+ 95us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:13.407296 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.407589 (+ 293us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.407663 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260806 03:54:13.407773 1922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.399407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:45029)
0806 03:54:13.399579 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.399618 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.400122 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.400331 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.400334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.400905 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:13.401105 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.401110 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.401745 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.401745 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.402635 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.402638 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.402956 (+ 318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.402957 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.403075 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.403508 (+ 433us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:13.403517 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:13.404924 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:13.406587 (+ 1663us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.406596 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.406598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.406827 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.407009 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.407010 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.407010 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.407122 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.407292 (+ 170us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:13.407295 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:13.407404 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.407476 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.407597 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":82,"threads_started":1}
I20260806 03:54:13.408285 1920 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38993
I20260806 03:54:13.408375 1920 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:13.408560 1920 heartbeater.cc:511] Master 127.12.15.126:38993 requested a full tablet report, sending...
I20260806 03:54:13.408952 1217 ts_manager.cc:194] Registered new tserver with Master: 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347)
I20260806 03:54:13.409328 1217 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:45029
I20260806 03:54:13.417595 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:54:13.424129 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.418852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.12.15.126:38993)
0806 03:54:13.418908 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.418910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.419005 (+ 95us) server_negotiation.cc:407] Connection header received
0806 03:54:13.419067 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.419068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.419078 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.419122 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:13.419652 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.420116 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.420812 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.420991 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.421044 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.422073 (+ 1029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:13.422074 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:13.422075 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:13.422082 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:13.423105 (+ 1023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.423403 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.423404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.423405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.423457 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.423612 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.423612 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.423613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.423761 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:13.423852 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.423966 (+ 114us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.424041 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:54:13.425568 1217 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:54:13.427268 1217 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:54:13.433694 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.429616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53962 (local address 127.12.15.67:37347)
0806 03:54:13.430238 (+ 622us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.430242 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.430251 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:54:13.430283 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.430285 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.430299 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.430420 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:13.430504 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.431176 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.432566 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.433291 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.433369 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.433438 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.433573 (+ 135us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.433615 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":587,"thread_start_us":86,"threads_started":1}
I20260806 03:54:13.433701 1938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.429380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:35647 (local address 127.0.0.1:35830)
0806 03:54:13.429550 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.429592 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.429659 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.430152 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.430153 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.430155 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:13.430226 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.430228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.431562 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.431563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.432609 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.432611 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.433457 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.433459 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.433529 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.433533 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.433563 (+ 30us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.433612 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":4,"thread_start_us":84,"threads_started":1}
I20260806 03:54:13.433709 1942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.429397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:37347 (local address 127.0.0.1:53962)
0806 03:54:13.429777 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.429786 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.429821 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.430399 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.430400 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.430402 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:13.430454 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.430456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.431260 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.431261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.432482 (+ 1221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.432485 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.433379 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.433380 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.433458 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.433473 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.433549 (+ 76us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.433608 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":88,"threads_started":1}
I20260806 03:54:13.434197 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.429789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35830 (local address 127.12.15.65:35647)
0806 03:54:13.429955 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.429959 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.429970 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:54:13.430004 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.430006 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.430018 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.430309 (+ 291us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:13.430397 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.431451 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.432677 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.433366 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.433815 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.434021 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.434049 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.434093 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":83,"threads_started":1}
I20260806 03:54:13.434546 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.429380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:41783 (local address 127.0.0.1:40288)
0806 03:54:13.429608 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.429618 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.429672 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.430102 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.430106 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.430108 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:13.430226 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.430230 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.432674 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.432674 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.433310 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.433311 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.434220 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.434221 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.434378 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.434383 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.434437 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.434476 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"mutex_wait_us":22,"thread_start_us":59,"threads_started":1}
I20260806 03:54:13.434844 1943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.429582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40288 (local address 127.12.15.66:41783)
0806 03:54:13.429927 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.429932 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.429940 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:54:13.429971 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.429972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.430010 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.430082 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:13.430315 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.431464 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.433390 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.434119 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.434200 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.434259 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.434734 (+ 475us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.434782 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":81,"threads_started":1}
I20260806 03:54:13.435076 1823 tablet_service.cc:1467] Processing CreateTablet for tablet 91e1215074044b26b297e2c42a19e6aa (DEFAULT_TABLE table=test-table [id=b960506b64f247c88bee30459a928eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:13.435423 1823 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91e1215074044b26b297e2c42a19e6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:13.435474 1616 tablet_service.cc:1467] Processing CreateTablet for tablet 91e1215074044b26b297e2c42a19e6aa (DEFAULT_TABLE table=test-table [id=b960506b64f247c88bee30459a928eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:13.435791 1616 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91e1215074044b26b297e2c42a19e6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:13.438081 1948 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: Bootstrap starting.
I20260806 03:54:13.438433 1949 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Bootstrap starting.
I20260806 03:54:13.438468 1411 tablet_service.cc:1467] Processing CreateTablet for tablet 91e1215074044b26b297e2c42a19e6aa (DEFAULT_TABLE table=test-table [id=b960506b64f247c88bee30459a928eb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:13.438817 1411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 91e1215074044b26b297e2c42a19e6aa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:13.438884 1948 tablet_bootstrap.cc:654] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:13.439086 1949 tablet_bootstrap.cc:654] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:13.439380 1948 log.cc:824] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:13.439617 1949 log.cc:824] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:13.440119 1948 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: No bootstrap required, opened a new log
I20260806 03:54:13.440183 1948 ts_tablet_manager.cc:1397] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:13.440351 1949 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: No bootstrap required, opened a new log
I20260806 03:54:13.440425 1949 ts_tablet_manager.cc:1397] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:54:13.441736 1948 raft_consensus.cc:348] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.441880 1948 raft_consensus.cc:374] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:13.441919 1948 raft_consensus.cc:729] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 516e1c592f424ebd8ec8487398ef7cc1, State: Initialized, Role: FOLLOWER
I20260806 03:54:13.442051 1948 consensus_queue.cc:260] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.442193 1952 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Bootstrap starting.
I20260806 03:54:13.442377 1948 ts_tablet_manager.cc:1428] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:54:13.442510 1920 heartbeater.cc:503] Master 127.12.15.126:38993 was elected leader, sending a full tablet report...
I20260806 03:54:13.442510 1949 raft_consensus.cc:348] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.442667 1949 raft_consensus.cc:374] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:13.442705 1949 raft_consensus.cc:729] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6351d2ff6f74e1e82fb88fbe6b5bc2c, State: Initialized, Role: FOLLOWER
I20260806 03:54:13.442818 1949 consensus_queue.cc:260] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.442878 1952 tablet_bootstrap.cc:654] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:13.443076 1949 ts_tablet_manager.cc:1428] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:54:13.443234 1719 heartbeater.cc:503] Master 127.12.15.126:38993 was elected leader, sending a full tablet report...
I20260806 03:54:13.443394 1952 log.cc:824] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:13.444061 1952 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: No bootstrap required, opened a new log
I20260806 03:54:13.444115 1952 ts_tablet_manager.cc:1397] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:13.445454 1952 raft_consensus.cc:348] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.445582 1952 raft_consensus.cc:374] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:13.445604 1952 raft_consensus.cc:729] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5268740ff3d342a9a1dc1d7a9dc63c82, State: Initialized, Role: FOLLOWER
I20260806 03:54:13.445677 1952 consensus_queue.cc:260] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.445870 1952 ts_tablet_manager.cc:1428] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:13.445936 1505 heartbeater.cc:503] Master 127.12.15.126:38993 was elected leader, sending a full tablet report...
W20260806 03:54:13.491145 1720 tablet.cc:2369] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:54:13.509168 1956 raft_consensus.cc:482] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:13.509331 1956 raft_consensus.cc:504] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.509706 1956 leader_election.cc:288] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647), 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347)
I20260806 03:54:13.512409 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.509876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:35647 (local address 127.12.15.66:46079)
0806 03:54:13.509914 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.509925 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.509966 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.510188 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.510190 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.510192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:13.510264 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.510266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.510889 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.510890 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.511623 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.511625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.512159 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.512160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.512244 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.512247 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.512292 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.512333 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19}
I20260806 03:54:13.512410 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.509977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:46079 (local address 127.12.15.65:35647)
0806 03:54:13.510037 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.510039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.510051 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:54:13.510080 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.510081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.510093 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.510179 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:13.510333 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.510814 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.511711 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.512085 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.512149 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.512159 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.512307 (+ 148us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.512341 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:54:13.512576 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.509989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:42473 (local address 127.12.15.67:37347)
0806 03:54:13.510044 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.510046 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.510275 (+ 229us) server_negotiation.cc:407] Connection header received
0806 03:54:13.510320 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.510321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.510330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.510373 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:13.510520 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.510984 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.511883 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.512259 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.512321 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.512329 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.512475 (+ 146us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.512513 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:54:13.512586 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.509876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:37347 (local address 127.12.15.66:42473)
0806 03:54:13.510193 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.510211 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.510266 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.510383 (+ 117us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.510383 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.510385 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:13.510454 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.510455 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.511060 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.511061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.511798 (+ 737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.511800 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.512335 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.512336 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.512414 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.512417 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.512462 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.512500 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"mutex_wait_us":138,"thread_start_us":129,"threads_started":1}
I20260806 03:54:13.512789 1445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91e1215074044b26b297e2c42a19e6aa" candidate_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" is_pre_election: true
I20260806 03:54:13.512902 1851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91e1215074044b26b297e2c42a19e6aa" candidate_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "516e1c592f424ebd8ec8487398ef7cc1" is_pre_election: true
I20260806 03:54:13.512962 1445 raft_consensus.cc:2454] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6351d2ff6f74e1e82fb88fbe6b5bc2c in term 0.
I20260806 03:54:13.513065 1851 raft_consensus.cc:2454] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6351d2ff6f74e1e82fb88fbe6b5bc2c in term 0.
I20260806 03:54:13.513223 1542 leader_election.cc:302] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5268740ff3d342a9a1dc1d7a9dc63c82, e6351d2ff6f74e1e82fb88fbe6b5bc2c; no voters:
I20260806 03:54:13.513350 1956 raft_consensus.cc:2781] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:54:13.513409 1956 raft_consensus.cc:482] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:54:13.513432 1956 raft_consensus.cc:3037] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:13.514166 1956 raft_consensus.cc:504] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.514295 1956 leader_election.cc:288] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [CANDIDATE]: Term 1 election: Requested vote from peers 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647), 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347)
I20260806 03:54:13.514451 1851 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91e1215074044b26b297e2c42a19e6aa" candidate_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "516e1c592f424ebd8ec8487398ef7cc1"
I20260806 03:54:13.514528 1851 raft_consensus.cc:3037] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:13.515218 1851 raft_consensus.cc:2454] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6351d2ff6f74e1e82fb88fbe6b5bc2c in term 1.
I20260806 03:54:13.515322 1445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "91e1215074044b26b297e2c42a19e6aa" candidate_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82"
I20260806 03:54:13.515386 1539 leader_election.cc:302] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 516e1c592f424ebd8ec8487398ef7cc1, e6351d2ff6f74e1e82fb88fbe6b5bc2c; no voters:
I20260806 03:54:13.515404 1445 raft_consensus.cc:3037] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:13.515473 1956 raft_consensus.cc:2781] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:13.515642 1956 raft_consensus.cc:686] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 LEADER]: Becoming Leader. State: Replica: e6351d2ff6f74e1e82fb88fbe6b5bc2c, State: Running, Role: LEADER
I20260806 03:54:13.515756 1956 consensus_queue.cc:237] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.516322 1445 raft_consensus.cc:2454] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6351d2ff6f74e1e82fb88fbe6b5bc2c in term 1.
I20260806 03:54:13.516618 1216 catalog_manager.cc:5495] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c reported cstate change: term changed from 0 to 1, leader changed from <none> to e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66). New cstate: current_term: 1 leader_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } health_report { overall_health: UNKNOWN } } }
I20260806 03:54:13.523984 1943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.522055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40292 (local address 127.12.15.66:41783)
0806 03:54:13.522093 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.522095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.522103 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:54:13.522169 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.522170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.522189 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.522250 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:54:13.522351 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.522778 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.523437 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.523612 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.523655 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.523754 (+ 99us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:54:13.523839 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:54:13.523891 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.523903 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.523918 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:54:13.525663 1851 raft_consensus.cc:1261] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 1 FOLLOWER]: Refusing update from remote peer e6351d2ff6f74e1e82fb88fbe6b5bc2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:13.525663 1445 raft_consensus.cc:1261] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Refusing update from remote peer e6351d2ff6f74e1e82fb88fbe6b5bc2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:13.526018 1963 consensus_queue.cc:1035] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:13.526155 1956 consensus_queue.cc:1035] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:13.528738 1968 mvcc.cc:205] Tried to move back new op lower bound from 7315408705638875136 to 7315408705601183744. Current Snapshot: MvccSnapshot[applied={T|T < 7315408705638875136}]
I20260806 03:54:13.530525 1217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:13.530625 1217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:13.531018 1969 mvcc.cc:205] Tried to move back new op lower bound from 7315408705638875136 to 7315408705601183744. Current Snapshot: MvccSnapshot[applied={T|T < 7315408705638875136}]
I20260806 03:54:13.531392 1217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Sending DeleteTablet for 3 replicas of tablet 91e1215074044b26b297e2c42a19e6aa
I20260806 03:54:13.531734 1411 tablet_service.cc:1514] Processing DeleteTablet for tablet 91e1215074044b26b297e2c42a19e6aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:13 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:35830
I20260806 03:54:13.531734 1823 tablet_service.cc:1514] Processing DeleteTablet for tablet 91e1215074044b26b297e2c42a19e6aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:13 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:53962
I20260806 03:54:13.531903 1616 tablet_service.cc:1514] Processing DeleteTablet for tablet 91e1215074044b26b297e2c42a19e6aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:13 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:40288
I20260806 03:54:13.531952 1983 tablet_replica.cc:330] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: stopping tablet replica
I20260806 03:54:13.532045 1983 raft_consensus.cc:2229] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:13.532135 1983 raft_consensus.cc:2258] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:13.532123 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 1311
I20260806 03:54:13.536470 1982 tablet_replica.cc:330] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: stopping tablet replica
I20260806 03:54:13.536580 1982 raft_consensus.cc:2229] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:13.536695 1982 raft_consensus.cc:2258] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:13.540349 1982 ts_tablet_manager.cc:1905] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:13.541497 1984 tablet_replica.cc:330] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: stopping tablet replica
I20260806 03:54:13.541639 1984 raft_consensus.cc:2229] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:54:13.541795 1984 raft_consensus.cc:2258] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:13.541960 1982 ts_tablet_manager.cc:1918] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:13.542037 1982 log.cc:1196] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/91e1215074044b26b297e2c42a19e6aa
I20260806 03:54:13.542361 1982 ts_tablet_manager.cc:1939] T 91e1215074044b26b297e2c42a19e6aa P 516e1c592f424ebd8ec8487398ef7cc1: Deleting consensus metadata
I20260806 03:54:13.543406 1188 catalog_manager.cc:4841] TS 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347): tablet 91e1215074044b26b297e2c42a19e6aa (table test-table [id=b960506b64f247c88bee30459a928eb5]) successfully deleted
I20260806 03:54:13.546422 1984 ts_tablet_manager.cc:1905] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260806 03:54:13.547366 1191 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:35647 (error 108)
I20260806 03:54:13.547672 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 1510
I20260806 03:54:13.548015 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.547904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:35647 (local address 127.0.0.1:35846)
0806 03:54:13.547950 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.547981 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 03:54:13.548015 1984 ts_tablet_manager.cc:1918] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:13.548082 1984 log.cc:1196] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/91e1215074044b26b297e2c42a19e6aa
W20260806 03:54:13.548281 1191 catalog_manager.cc:4584] TS 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 91e1215074044b26b297e2c42a19e6aa: Network error: Client connection negotiation failed: client connection to 127.12.15.65:35647: connect: Connection refused (error 111)
I20260806 03:54:13.555896 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 1727
I20260806 03:54:13.555987 1940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.555871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:41783 (local address 127.0.0.1:40302)
0806 03:54:13.555922 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.555955 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.66:41783: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:54:13.562112 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 1163
I20260806 03:54:13.574771 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38993
--webserver_interface=127.12.15.126
--webserver_port=39891
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:38993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:13.656399 1986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:13.656601 1986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:13.656622 1986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:13.657697 1986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:13.657732 1986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:13.657744 1986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:13.657755 1986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:13.657769 1986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:13.659195 1986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:38993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=39891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:13.659380 1986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:13.659559 1986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:13.669916 1995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.670071 1986 server_base.cc:1034] running on GCE node
W20260806 03:54:13.669934 1996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.669934 1999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.670473 1986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:13.670789 1986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:13.671926 1986 hybrid_clock.cc:648] HybridClock initialized: now 1785988453671910 us; error 28 us; skew 500 ppm
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:13.674361 1986 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:13.674798 1986 webserver.cc:460] Webserver started at http://127.12.15.126:39891/ using document root <none> and password file <none>
I20260806 03:54:13.675026 1986 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:13.675083 1986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:13.676414 1986 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:54:13.677016 2009 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.677158 1986 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:54:13.677248 1986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "0e320a4526f24e91845b15e1f8e42f1a"
format_stamp: "Formatted at 2026-08-06 03:54:12 on dist-test-slave-bqcl"
server_key: "4d17d81d4dcfc23850b7359e4843d663"
server_key_iv: "08c1a32342fca81f6361ee64c0c60f36"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.677640 1986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:13.687235 1986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:13.688868 1986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:13.688997 1986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:13.692875 1986 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:38993
I20260806 03:54:13.692914 2088 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:38993 every 8 connection(s)
I20260806 03:54:13.693269 1986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:13.694952 2089 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:54:13.695564 2089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Bootstrap starting.
I20260806 03:54:13.699690 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 1986
I20260806 03:54:13.699954 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:35647
--local_ip_for_outbound_sockets=127.12.15.65
--tserver_master_addrs=127.12.15.126:38993
--webserver_port=33529
--webserver_interface=127.12.15.65
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:13.703943 2089 log.cc:824] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:13.705849 2089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:54:13.706110 2089 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Bootstrap complete.
I20260806 03:54:13.708285 2089 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:13.708509 2089 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e320a4526f24e91845b15e1f8e42f1a, State: Initialized, Role: FOLLOWER
I20260806 03:54:13.708700 2089 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:13.708770 2089 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:54:13.708818 2089 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:54:13.708870 2089 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:54:13.709854 2089 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:13.709952 2089 leader_election.cc:302] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e320a4526f24e91845b15e1f8e42f1a; no voters:
I20260806 03:54:13.710137 2089 leader_election.cc:288] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:54:13.710222 2093 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:54:13.710426 2089 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:13.710427 2093 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 2 LEADER]: Becoming Leader. State: Replica: 0e320a4526f24e91845b15e1f8e42f1a, State: Running, Role: LEADER
I20260806 03:54:13.710587 2093 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:13.710976 2094 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "0e320a4526f24e91845b15e1f8e42f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } } }
I20260806 03:54:13.711292 2094 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:13.711438 2095 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e320a4526f24e91845b15e1f8e42f1a. Latest consensus state: current_term: 2 leader_uuid: "0e320a4526f24e91845b15e1f8e42f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } } }
I20260806 03:54:13.711483 2107 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:13.711583 2095 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:13.712518 2107 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:54:13.712623 2107 catalog_manager.cc:1259] Loaded cluster ID: dfb9522ffd75459cb200af9dfc58c67c
I20260806 03:54:13.712653 2107 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:54:13.712875 2107 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260806 03:54:13.717955 2107 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260806 03:54:13.720337 2107 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:54:13.720554 2107 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Loaded TSK: 0
I20260806 03:54:13.720777 2107 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:54:13.785969 2091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:13.786206 2091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:13.786234 2091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:13.787365 2091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:13.787406 2091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:13.787431 2091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:54:13.788875 2091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:35647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=33529
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2091
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:13.789093 2091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:13.789321 2091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:13.795536 2126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.795513 2125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.795527 2091 server_base.cc:1034] running on GCE node
W20260806 03:54:13.795825 2129 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.796042 2091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:13.796278 2091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:13.797437 2091 hybrid_clock.cc:648] HybridClock initialized: now 1785988453797422 us; error 31 us; skew 500 ppm
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:13.800100 2091 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:54:13.800535 2091 webserver.cc:460] Webserver started at http://127.12.15.65:33529/ using document root <none> and password file <none>
I20260806 03:54:13.800760 2091 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:13.800815 2091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:13.802250 2091 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:13.802914 2139 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.803090 2091 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:13.803177 2091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "5268740ff3d342a9a1dc1d7a9dc63c82"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "b84bbe50a75ee2df7e4c7c0a8b7983da"
server_key_iv: "40789113fa0f328a81943f8151e765b1"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.803524 2091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:13.817195 2091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:13.819439 2091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:13.819577 2091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:13.819808 2091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:13.820283 2149 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 03:54:13.821679 2091 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:54:13.821739 2091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 03:54:13.821765 2091 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:54:13.822471 2091 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:54:13.822517 2091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 03:54:13.822578 2149 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Bootstrap starting.
I20260806 03:54:13.830085 2091 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:35647
I20260806 03:54:13.830623 2091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:54:13.833870 2149 log.cc:824] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:13.834198 2300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:35647 every 8 connection(s)
I20260806 03:54:13.835892 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 2091
I20260806 03:54:13.836193 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:41783
--local_ip_for_outbound_sockets=127.12.15.66
--tserver_master_addrs=127.12.15.126:38993
--webserver_port=39791
--webserver_interface=127.12.15.66
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:13.838289 2149 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:54:13.838654 2149 tablet_bootstrap.cc:492] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Bootstrap complete.
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:13.841091 2149 ts_tablet_manager.cc:1397] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Time spent bootstrapping tablet: real 0.019s user 0.009s sys 0.006s
I20260806 03:54:13.842453 2149 raft_consensus.cc:348] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.842829 2149 raft_consensus.cc:729] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5268740ff3d342a9a1dc1d7a9dc63c82, State: Initialized, Role: FOLLOWER
I20260806 03:54:13.843024 2149 consensus_queue.cc:260] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } } peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } }
I20260806 03:54:13.843317 2149 ts_tablet_manager.cc:1428] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:13.843899 2304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.831397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:43339)
0806 03:54:13.831658 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.831697 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.832289 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.833037 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.833043 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.833470 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:13.833773 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.833781 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.835406 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.835407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.836512 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.836516 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.837332 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.837333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.837476 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.838047 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:13.838056 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:13.839811 (+ 1755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:13.841969 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.841977 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.841979 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.842316 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.842631 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.842632 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.842633 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.842767 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.843248 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:13.843257 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:13.843420 (+ 163us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.843513 (+ 93us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.843687 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":91,"threads_started":1}
I20260806 03:54:13.844493 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.831536 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:43339 (local address 127.12.15.126:38993)
0806 03:54:13.832508 (+ 972us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.832517 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.832582 (+ 65us) server_negotiation.cc:407] Connection header received
0806 03:54:13.832737 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.832743 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.832800 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.832934 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:13.833920 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.835292 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.836616 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.837199 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.837763 (+ 564us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.839918 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:13.839923 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:13.839923 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:13.839935 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:13.841868 (+ 1933us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.842425 (+ 557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.842428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.842429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.842550 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.842846 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.842847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.842847 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.843142 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:13.844056 (+ 914us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.844138 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.844296 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":919,"thread_start_us":93,"threads_started":1}
I20260806 03:54:13.845211 2302 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38993
I20260806 03:54:13.845325 2302 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:13.845500 2302 heartbeater.cc:511] Master 127.12.15.126:38993 requested a full tablet report, sending...
I20260806 03:54:13.846346 2033 ts_manager.cc:194] Registered new tserver with Master: 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647)
I20260806 03:54:13.847548 2033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:43339
I20260806 03:54:13.848379 2302 heartbeater.cc:503] Master 127.12.15.126:38993 was elected leader, sending a full tablet report...
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:54:13.854719 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.847036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35860 (local address 127.12.15.65:35647)
0806 03:54:13.847168 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.847172 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.847294 (+ 122us) server_negotiation.cc:407] Connection header received
0806 03:54:13.847376 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.847379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.847394 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.847455 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:13.848020 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.849031 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.849736 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.850211 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.850282 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.852284 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:13.852291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:13.852291 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:13.852297 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:13.853589 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.853992 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.853993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.853994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.854041 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.854244 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.854245 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.854245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.854368 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:13.854427 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.854578 (+ 151us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.854649 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":60,"threads_started":1}
I20260806 03:54:13.854763 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.846967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:35647 (local address 127.0.0.1:35860)
0806 03:54:13.847181 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.847226 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.847291 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.847463 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.847467 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.847764 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:13.847814 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.847818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.849109 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.849110 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.849657 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.849659 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.850289 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.850290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.850342 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.850765 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:13.850769 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:13.852191 (+ 1422us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:13.853661 (+ 1470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.853668 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.853670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.853925 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.854092 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.854093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.854093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.854180 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.854421 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:13.854427 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:13.854476 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.854553 (+ 77us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.854664 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":88,"threads_started":1}
I20260806 03:54:13.855334 2201 tablet_service.cc:1514] Processing DeleteTablet for tablet 91e1215074044b26b297e2c42a19e6aa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:13 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:35860
I20260806 03:54:13.855794 2315 tablet_replica.cc:330] stopping tablet replica
I20260806 03:54:13.855906 2315 raft_consensus.cc:2229] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:13.855967 2315 raft_consensus.cc:2258] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:13.856262 2315 ts_tablet_manager.cc:1905] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:13.858371 2315 ts_tablet_manager.cc:1918] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:13.858448 2315 log.cc:1196] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/91e1215074044b26b297e2c42a19e6aa
I20260806 03:54:13.858678 2315 ts_tablet_manager.cc:1939] T 91e1215074044b26b297e2c42a19e6aa P 5268740ff3d342a9a1dc1d7a9dc63c82: Deleting consensus metadata
I20260806 03:54:13.859146 2013 catalog_manager.cc:4841] TS 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647): tablet 91e1215074044b26b297e2c42a19e6aa (table test-table [id=b960506b64f247c88bee30459a928eb5]) successfully deleted
W20260806 03:54:13.924059 2310 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:13.924270 2310 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:13.924293 2310 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:13.925453 2310 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:13.925495 2310 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:13.925522 2310 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:54:13.927048 2310 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:41783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=39791
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2310
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:13.927287 2310 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:13.927501 2310 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:54:13.932406 2310 server_base.cc:1034] running on GCE node
W20260806 03:54:13.932336 2327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.932310 2323 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:13.932333 2324 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:13.932832 2310 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:13.933036 2310 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:13.934167 2310 hybrid_clock.cc:648] HybridClock initialized: now 1785988453934158 us; error 28 us; skew 500 ppm
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:13.936475 2310 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:54:13.936882 2310 webserver.cc:460] Webserver started at http://127.12.15.66:39791/ using document root <none> and password file <none>
I20260806 03:54:13.937084 2310 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:13.937147 2310 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:13.938452 2310 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:54:13.939028 2338 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.939180 2310 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:54:13.939271 2310 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "37cc06ea59e37912ff4925cb4b8e9fad"
server_key_iv: "49c1e0a0cce28d3f746aca5fe49e96e1"
server_key_version: "encryptionkey@0"
I20260806 03:54:13.939651 2310 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:13.950919 2310 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:13.952664 2310 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:13.952790 2310 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:13.952968 2310 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:13.953377 2348 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260806 03:54:13.954823 2348 ts_tablet_manager.cc:1852] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260806 03:54:13.955111 2348 ts_tablet_manager.cc:1905] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:13.957197 2348 ts_tablet_manager.cc:1918] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:13.957279 2348 log.cc:1196] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/91e1215074044b26b297e2c42a19e6aa
I20260806 03:54:13.957525 2348 ts_tablet_manager.cc:1939] T 91e1215074044b26b297e2c42a19e6aa P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Deleting consensus metadata
I20260806 03:54:13.957712 2310 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260806 03:54:13.957759 2310 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.004s user 0.000s sys 0.000s
I20260806 03:54:13.957782 2310 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:13.957794 2310 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:13.963234 2310 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:41783
I20260806 03:54:13.963299 2497 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:41783 every 8 connection(s)
I20260806 03:54:13.963562 2310 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:54:13 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988453, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:13.971755 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.964198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:55803 (local address 127.12.15.126:38993)
0806 03:54:13.964258 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:13.964260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:13.964458 (+ 198us) server_negotiation.cc:407] Connection header received
0806 03:54:13.964959 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:13.964961 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:13.964974 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:13.965027 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:13.965649 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.966200 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.967161 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.967367 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.967424 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.969381 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:13.969383 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:13.969383 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:13.969391 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:13.970500 (+ 1109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.970864 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.970866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.970866 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.970921 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:13.971132 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:13.971133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:13.971133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:13.971244 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:13.971301 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:13.971556 (+ 255us) server_negotiation.cc:299] Negotiation successful
0806 03:54:13.971649 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:54:13.971915 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:13.964172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:55803)
0806 03:54:13.964342 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:13.964382 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:13.964861 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:13.965099 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:13.965104 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:13.965369 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:13.965556 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.965560 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.966282 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.966283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.967079 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:13.967081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:13.967436 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:13.967437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:13.967555 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:13.967985 (+ 430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:13.967993 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:13.969296 (+ 1303us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:13.970571 (+ 1275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.970578 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.970580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.970797 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.970975 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:13.970975 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:13.970976 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:13.971076 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:13.971300 (+ 224us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:13.971307 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:13.971438 (+ 131us) client_negotiation.cc:769] Sending connection context
0806 03:54:13.971526 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 03:54:13.971679 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260806 03:54:13.972376 2499 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38993
I20260806 03:54:13.972505 2499 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:13.972681 2499 heartbeater.cc:511] Master 127.12.15.126:38993 requested a full tablet report, sending...
I20260806 03:54:13.973033 2033 ts_manager.cc:194] Registered new tserver with Master: e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783)
I20260806 03:54:13.973320 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 2310
I20260806 03:54:13.973374 2033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:55803
I20260806 03:54:13.973611 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:37347
--local_ip_for_outbound_sockets=127.12.15.67
--tserver_master_addrs=127.12.15.126:38993
--webserver_port=42835
--webserver_interface=127.12.15.67
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:14.056632 2502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:14.056859 2502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:14.056892 2502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:14.058022 2502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:14.058063 2502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:14.058099 2502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:54:14.059553 2502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:37347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=42835
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:14.059813 2502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:14.060034 2502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:14.064965 2513 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:14.064936 2515 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:14.065160 2502 server_base.cc:1034] running on GCE node
W20260806 03:54:14.064958 2512 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:14.065411 2502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:14.065663 2502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:14.066802 2502 hybrid_clock.cc:648] HybridClock initialized: now 1785988454066781 us; error 31 us; skew 500 ppm
Aug 06 03:54:14 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988454, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:14.069185 2502 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:54:14.069615 2502 webserver.cc:460] Webserver started at http://127.12.15.67:42835/ using document root <none> and password file <none>
I20260806 03:54:14.069828 2502 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:14.069895 2502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:14.071273 2502 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:14.071820 2523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.071969 2502 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:14.072054 2502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "516e1c592f424ebd8ec8487398ef7cc1"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "6aae94d1091a762db0f14321f1c0771a"
server_key_iv: "dd42fd4723ba93965157ad3050b6279c"
server_key_version: "encryptionkey@0"
I20260806 03:54:14.072409 2502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:14.083067 2502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:14.084754 2502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:14.084888 2502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:14.085090 2502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:14.085403 2502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:14.085445 2502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.085474 2502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:14.085500 2502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.091329 2502 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:37347
I20260806 03:54:14.091392 2695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:37347 every 8 connection(s)
I20260806 03:54:14.091698 2502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:54:14 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988454, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:14.099063 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 2502
I20260806 03:54:14.100554 2305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.092349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:45945 (local address 127.12.15.126:38993)
0806 03:54:14.092429 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:14.092431 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:14.092628 (+ 197us) server_negotiation.cc:407] Connection header received
0806 03:54:14.093118 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:14.093119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:14.093130 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:14.093169 (+ 39us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:14.094080 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.094618 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.095580 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.095769 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.095813 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.097763 (+ 1950us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:14.097764 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:14.097765 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:14.097771 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:14.099135 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:14.099622 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:14.099624 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:14.099624 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:14.099694 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:14.099947 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:14.099948 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:14.099948 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:14.100094 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:14.100171 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:14.100407 (+ 236us) server_negotiation.cc:299] Negotiation successful
0806 03:54:14.100485 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:54:14.100785 2699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.092334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:45945)
0806 03:54:14.092496 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.092533 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:14.093022 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:14.093233 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:14.093236 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:14.093817 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:14.094011 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.094016 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.094696 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.094697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.095518 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.095520 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.095821 (+ 301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.095821 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.095917 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.096351 (+ 434us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:14.096359 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:14.097663 (+ 1304us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:14.099228 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:14.099236 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:14.099238 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:14.099546 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:14.099752 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:14.099753 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:14.099753 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:14.099889 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:14.100158 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:14.100161 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:14.100281 (+ 120us) client_negotiation.cc:769] Sending connection context
0806 03:54:14.100364 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 03:54:14.100534 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":85,"threads_started":1}
I20260806 03:54:14.101239 2696 heartbeater.cc:347] Connected to a master server at 127.12.15.126:38993
I20260806 03:54:14.101347 2696 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:14.101545 2696 heartbeater.cc:511] Master 127.12.15.126:38993 requested a full tablet report, sending...
I20260806 03:54:14.101951 2033 ts_manager.cc:194] Registered new tserver with Master: 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347)
I20260806 03:54:14.102360 2033 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:45945
I20260806 03:54:14.105363 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.099447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50702 (local address 127.12.15.126:38993)
0806 03:54:14.099553 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:14.099555 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:14.099653 (+ 98us) server_negotiation.cc:407] Connection header received
0806 03:54:14.099723 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:14.099724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:14.099738 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:14.099797 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:14.100405 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.101103 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.102058 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.102330 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.102381 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.103470 (+ 1089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:14.103471 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:14.103472 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:14.103478 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:14.104529 (+ 1051us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:14.104815 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:14.104815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:14.104815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:14.104861 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:14.104982 (+ 121us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:14.104983 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:14.104983 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:14.105066 (+ 83us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:14.105186 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:14.105235 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 03:54:14.105297 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83,"thread_start_us":51,"threads_started":1}
I20260806 03:54:14.106042 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:54:14.108741 2700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.106381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50706 (local address 127.12.15.126:38993)
0806 03:54:14.106425 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:14.106426 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:14.106529 (+ 103us) server_negotiation.cc:407] Connection header received
0806 03:54:14.106594 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:14.106594 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:14.106634 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:14.106693 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:54:14.106814 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.107289 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.108021 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.108235 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.108287 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.108440 (+ 153us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:54:14.108556 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:54:14.108630 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:14.108650 (+ 20us) server_negotiation.cc:299] Negotiation successful
0806 03:54:14.108673 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:54:14.108996 2033 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:50706:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:54:14.110780 2033 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:54:14.112777 2201 tablet_service.cc:1467] Processing CreateTablet for tablet c8b478be25c54200ae5672255278685a (DEFAULT_TABLE table=test-table [id=b7ebf7a5fada4c9e80fb869363ef9409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:14.112995 2201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8b478be25c54200ae5672255278685a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:14.114651 2149 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Bootstrap starting.
I20260806 03:54:14.115001 2149 tablet_bootstrap.cc:654] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:14.115581 2149 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: No bootstrap required, opened a new log
I20260806 03:54:14.115635 2149 ts_tablet_manager.cc:1397] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:14.115993 2149 raft_consensus.cc:348] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.116087 2149 raft_consensus.cc:374] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:14.116142 2149 raft_consensus.cc:729] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5268740ff3d342a9a1dc1d7a9dc63c82, State: Initialized, Role: FOLLOWER
I20260806 03:54:14.116192 2149 consensus_queue.cc:260] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.116286 2149 ts_tablet_manager.cc:1428] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:14.116297 2703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.112547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:37347 (local address 127.0.0.1:53972)
0806 03:54:14.112701 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.112710 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:14.112748 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:14.113302 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:14.113302 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:14.113304 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:14.113338 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.113338 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.114184 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.114185 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.115178 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.115182 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.116054 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.116055 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.116118 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.116142 (+ 24us) client_negotiation.cc:769] Sending connection context
0806 03:54:14.116177 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 03:54:14.116229 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":99,"threads_started":1}
I20260806 03:54:14.116472 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.112513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:41783 (local address 127.0.0.1:40310)
0806 03:54:14.112557 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.112568 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:14.112602 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:14.113189 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:14.113190 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:14.113192 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:14.113234 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.113235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.114490 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.114491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.115496 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.115499 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.116248 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.116249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.116325 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.116330 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:54:14.116362 (+ 32us) client_negotiation.cc:240] Negotiation successful
0806 03:54:14.116404 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260806 03:54:14.116671 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.112794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53972 (local address 127.12.15.67:37347)
0806 03:54:14.113028 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:14.113031 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:14.113042 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:54:14.113081 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:14.113083 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:14.113098 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:14.113174 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:14.113397 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.114062 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.115254 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.115950 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.116452 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.116526 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:14.116555 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 03:54:14.116595 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
I20260806 03:54:14.116816 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.112621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40310 (local address 127.12.15.66:41783)
0806 03:54:14.113018 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:14.113023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:14.113036 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:54:14.113078 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:14.113080 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:14.113096 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:14.113182 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:14.113300 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.114354 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.115582 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.116153 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.116580 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.116654 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:14.116679 (+ 25us) server_negotiation.cc:299] Negotiation successful
0806 03:54:14.116729 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":111,"threads_started":1}
I20260806 03:54:14.117363 2592 tablet_service.cc:1467] Processing CreateTablet for tablet c8b478be25c54200ae5672255278685a (DEFAULT_TABLE table=test-table [id=b7ebf7a5fada4c9e80fb869363ef9409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:14.117427 2403 tablet_service.cc:1467] Processing CreateTablet for tablet c8b478be25c54200ae5672255278685a (DEFAULT_TABLE table=test-table [id=b7ebf7a5fada4c9e80fb869363ef9409]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:14.117645 2592 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8b478be25c54200ae5672255278685a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:14.117712 2403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8b478be25c54200ae5672255278685a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:14.119400 2348 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Bootstrap starting.
I20260806 03:54:14.120276 2348 tablet_bootstrap.cc:654] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:14.120595 2348 log.cc:824] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:14.120877 2707 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: Bootstrap starting.
I20260806 03:54:14.121304 2348 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: No bootstrap required, opened a new log
I20260806 03:54:14.121357 2348 ts_tablet_manager.cc:1397] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:54:14.121496 2707 tablet_bootstrap.cc:654] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:14.121914 2707 log.cc:824] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:14.122542 2707 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: No bootstrap required, opened a new log
I20260806 03:54:14.122596 2707 ts_tablet_manager.cc:1397] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:54:14.122823 2348 raft_consensus.cc:348] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.122952 2348 raft_consensus.cc:374] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:14.122985 2348 raft_consensus.cc:729] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6351d2ff6f74e1e82fb88fbe6b5bc2c, State: Initialized, Role: FOLLOWER
I20260806 03:54:14.123100 2348 consensus_queue.cc:260] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.123346 2348 ts_tablet_manager.cc:1428] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:14.123401 2499 heartbeater.cc:503] Master 127.12.15.126:38993 was elected leader, sending a full tablet report...
I20260806 03:54:14.124119 2707 raft_consensus.cc:348] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.124266 2707 raft_consensus.cc:374] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:14.124312 2707 raft_consensus.cc:729] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 516e1c592f424ebd8ec8487398ef7cc1, State: Initialized, Role: FOLLOWER
I20260806 03:54:14.124414 2707 consensus_queue.cc:260] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.124639 2707 ts_tablet_manager.cc:1428] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:14.124675 2696 heartbeater.cc:503] Master 127.12.15.126:38993 was elected leader, sending a full tablet report...
I20260806 03:54:14.154541 2711 raft_consensus.cc:482] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:14.154659 2711 raft_consensus.cc:504] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.155041 2711 leader_election.cc:288] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347), 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647)
I20260806 03:54:14.157811 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.155307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:45503 (local address 127.12.15.67:37347)
0806 03:54:14.155356 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:14.155358 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:14.155369 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:54:14.155396 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:14.155397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:14.155407 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:14.155471 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:14.155633 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.156145 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.157085 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.157473 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.157526 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.157534 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:14.157707 (+ 173us) server_negotiation.cc:299] Negotiation successful
0806 03:54:14.157742 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:54:14.157817 2501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.155210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:37347 (local address 127.12.15.66:45503)
0806 03:54:14.155272 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.155284 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:14.155335 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:14.155484 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:14.155484 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:14.155486 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:14.155566 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.155568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.156213 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.156214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.156999 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.157001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.157544 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.157545 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.157624 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.157627 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:54:14.157692 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 03:54:14.157742 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 03:54:14.157946 2313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.155292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:33781 (local address 127.12.15.65:35647)
0806 03:54:14.155336 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:14.155338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:14.155488 (+ 150us) server_negotiation.cc:407] Connection header received
0806 03:54:14.155522 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:14.155523 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:14.155553 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:14.155625 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:14.155759 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.156286 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.157174 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.157646 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.157707 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.157784 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:14.157845 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 03:54:14.157885 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:54:14.158015 2713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.155234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:35647 (local address 127.12.15.66:33781)
0806 03:54:14.155413 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.155422 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:14.155477 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:14.155632 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:14.155633 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:14.155634 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:14.155697 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.155698 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.156363 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.156364 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.157088 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:14.157090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.157715 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.157716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:14.157783 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.157786 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:54:14.157840 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 03:54:14.157873 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":123,"threads_started":1}
I20260806 03:54:14.158108 2624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8b478be25c54200ae5672255278685a" candidate_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "516e1c592f424ebd8ec8487398ef7cc1" is_pre_election: true
I20260806 03:54:14.158329 2624 raft_consensus.cc:2454] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6351d2ff6f74e1e82fb88fbe6b5bc2c in term 0.
I20260806 03:54:14.158334 2237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8b478be25c54200ae5672255278685a" candidate_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" is_pre_election: true
I20260806 03:54:14.158469 2237 raft_consensus.cc:2454] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e6351d2ff6f74e1e82fb88fbe6b5bc2c in term 0.
I20260806 03:54:14.158579 2341 leader_election.cc:302] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 516e1c592f424ebd8ec8487398ef7cc1, e6351d2ff6f74e1e82fb88fbe6b5bc2c; no voters:
I20260806 03:54:14.158753 2711 raft_consensus.cc:2781] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:54:14.158813 2711 raft_consensus.cc:482] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:54:14.158835 2711 raft_consensus.cc:3037] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:14.159483 2711 raft_consensus.cc:504] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.159605 2711 leader_election.cc:288] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [CANDIDATE]: Term 1 election: Requested vote from peers 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347), 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647)
I20260806 03:54:14.159754 2624 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8b478be25c54200ae5672255278685a" candidate_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "516e1c592f424ebd8ec8487398ef7cc1"
I20260806 03:54:14.159765 2237 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8b478be25c54200ae5672255278685a" candidate_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82"
I20260806 03:54:14.159835 2237 raft_consensus.cc:3037] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:14.159829 2624 raft_consensus.cc:3037] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:14.160632 2237 raft_consensus.cc:2454] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6351d2ff6f74e1e82fb88fbe6b5bc2c in term 1.
I20260806 03:54:14.160730 2624 raft_consensus.cc:2454] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e6351d2ff6f74e1e82fb88fbe6b5bc2c in term 1.
I20260806 03:54:14.160799 2342 leader_election.cc:302] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5268740ff3d342a9a1dc1d7a9dc63c82, e6351d2ff6f74e1e82fb88fbe6b5bc2c; no voters:
I20260806 03:54:14.160889 2711 raft_consensus.cc:2781] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:14.161033 2711 raft_consensus.cc:686] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 LEADER]: Becoming Leader. State: Replica: e6351d2ff6f74e1e82fb88fbe6b5bc2c, State: Running, Role: LEADER
I20260806 03:54:14.161118 2711 consensus_queue.cc:237] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.161907 2033 catalog_manager.cc:5495] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c reported cstate change: term changed from 0 to 1, leader changed from <none> to e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66). New cstate: current_term: 1 leader_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } health_report { overall_health: UNKNOWN } } }
I20260806 03:54:14.176298 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.174038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40326 (local address 127.12.15.66:41783)
0806 03:54:14.174093 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:14.174095 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:14.174104 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:54:14.174183 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:14.174184 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:14.174203 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:14.174265 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:54:14.174381 (+ 116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.174847 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.175562 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:14.175758 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:14.175812 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:14.175971 (+ 159us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:54:14.176084 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:54:14.176153 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:14.176189 (+ 36us) server_negotiation.cc:299] Negotiation successful
0806 03:54:14.176209 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:54:14.178028 2624 raft_consensus.cc:1261] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 1 FOLLOWER]: Refusing update from remote peer e6351d2ff6f74e1e82fb88fbe6b5bc2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:14.178041 2237 raft_consensus.cc:1261] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Refusing update from remote peer e6351d2ff6f74e1e82fb88fbe6b5bc2c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:14.178369 2715 consensus_queue.cc:1035] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:14.178491 2711 consensus_queue.cc:1035] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:14.180631 2718 mvcc.cc:205] Tried to move back new op lower bound from 7315408708310753280 to 7315408708244447232. Current Snapshot: MvccSnapshot[applied={T|T < 7315408708310753280}]
I20260806 03:54:14.183341 2032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:50706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:14.183429 2032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:50706:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:14.184156 2032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Sending DeleteTablet for 3 replicas of tablet c8b478be25c54200ae5672255278685a
I20260806 03:54:14.184475 2403 tablet_service.cc:1514] Processing DeleteTablet for tablet c8b478be25c54200ae5672255278685a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:14 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:40310
I20260806 03:54:14.184535 2201 tablet_service.cc:1514] Processing DeleteTablet for tablet c8b478be25c54200ae5672255278685a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:14 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:35860
I20260806 03:54:14.184655 2315 tablet_replica.cc:330] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: stopping tablet replica
I20260806 03:54:14.184672 2592 tablet_service.cc:1514] Processing DeleteTablet for tablet c8b478be25c54200ae5672255278685a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:14 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:53972
I20260806 03:54:14.184721 2315 raft_consensus.cc:2229] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:14.184728 2732 tablet_replica.cc:330] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: stopping tablet replica
I20260806 03:54:14.184779 2315 raft_consensus.cc:2258] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:14.184823 2732 raft_consensus.cc:2229] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:54:14.184952 2732 raft_consensus.cc:2258] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:14.185046 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 2091
I20260806 03:54:14.185142 2315 ts_tablet_manager.cc:1905] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:14.185398 2732 ts_tablet_manager.cc:1905] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:14.187181 2732 ts_tablet_manager.cc:1918] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:14.187247 2732 log.cc:1196] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/c8b478be25c54200ae5672255278685a
I20260806 03:54:14.187494 2732 ts_tablet_manager.cc:1939] T c8b478be25c54200ae5672255278685a P e6351d2ff6f74e1e82fb88fbe6b5bc2c: Deleting consensus metadata
I20260806 03:54:14.187952 2014 catalog_manager.cc:4841] TS e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783): tablet c8b478be25c54200ae5672255278685a (table test-table [id=b7ebf7a5fada4c9e80fb869363ef9409]) successfully deleted
I20260806 03:54:14.194674 2734 tablet_replica.cc:330] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: stopping tablet replica
I20260806 03:54:14.194844 2734 raft_consensus.cc:2229] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:14.194949 2734 raft_consensus.cc:2258] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:14.195937 2734 ts_tablet_manager.cc:1905] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:14.197474 2734 ts_tablet_manager.cc:1918] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:14.197548 2734 log.cc:1196] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/c8b478be25c54200ae5672255278685a
I20260806 03:54:14.197804 2734 ts_tablet_manager.cc:1939] T c8b478be25c54200ae5672255278685a P 516e1c592f424ebd8ec8487398ef7cc1: Deleting consensus metadata
I20260806 03:54:14.198261 2010 catalog_manager.cc:4841] TS 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347): tablet c8b478be25c54200ae5672255278685a (table test-table [id=b7ebf7a5fada4c9e80fb869363ef9409]) successfully deleted
W20260806 03:54:14.201449 2013 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:35647 (error 108)
I20260806 03:54:14.201902 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 2310
I20260806 03:54:14.202004 2312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.201834 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:35647 (local address 127.0.0.1:35874)
0806 03:54:14.201906 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.201963 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:35647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260806 03:54:14.216800 2013 catalog_manager.cc:4584] TS 5268740ff3d342a9a1dc1d7a9dc63c82 (127.12.15.65:35647): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c8b478be25c54200ae5672255278685a: Network error: Client connection negotiation failed: client connection to 127.12.15.65:35647: connect: Connection refused (error 111)
I20260806 03:54:14.224221 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 2502
I20260806 03:54:14.232856 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 1986
I20260806 03:54:14.248474 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38993
--webserver_interface=127.12.15.126
--webserver_port=39891
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:38993
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:14.330093 2736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:14.330338 2736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:14.330359 2736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:14.331463 2736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:14.331496 2736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:14.331509 2736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:14.331521 2736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:14.331533 2736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:14.332971 2736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:38993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:38993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=39891
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:14.333155 2736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:14.333333 2736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:14.337736 2746 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:14.337738 2745 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:14.337814 2736 server_base.cc:1034] running on GCE node
W20260806 03:54:14.337808 2749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:14.338297 2736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:14.338562 2736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:14.339706 2736 hybrid_clock.cc:648] HybridClock initialized: now 1785988454339693 us; error 27 us; skew 500 ppm
Aug 06 03:54:14 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988454, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:14.342020 2736 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:14.342438 2736 webserver.cc:460] Webserver started at http://127.12.15.126:39891/ using document root <none> and password file <none>
I20260806 03:54:14.342628 2736 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:14.342676 2736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:14.343896 2736 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:54:14.344449 2760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.344599 2736 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:54:14.344681 2736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "0e320a4526f24e91845b15e1f8e42f1a"
format_stamp: "Formatted at 2026-08-06 03:54:12 on dist-test-slave-bqcl"
server_key: "4d17d81d4dcfc23850b7359e4843d663"
server_key_iv: "08c1a32342fca81f6361ee64c0c60f36"
server_key_version: "encryptionkey@0"
I20260806 03:54:14.344988 2736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:14.359732 2736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:14.361774 2736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:14.361909 2736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:14.365671 2736 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:38993
I20260806 03:54:14.365705 2832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:38993 every 8 connection(s)
I20260806 03:54:14.366044 2736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:14.367568 2833 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:54:14.368212 2833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Bootstrap starting.
I20260806 03:54:14.373484 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 2736
I20260806 03:54:14.373838 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:35647
--local_ip_for_outbound_sockets=127.12.15.65
--tserver_master_addrs=127.12.15.126:38993
--webserver_port=33529
--webserver_interface=127.12.15.65
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:14.376852 2833 log.cc:824] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:14.379043 2833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:54:14.379308 2833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a: Bootstrap complete.
I20260806 03:54:14.381321 2833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:14.381538 2833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0e320a4526f24e91845b15e1f8e42f1a, State: Initialized, Role: FOLLOWER
I20260806 03:54:14.381685 2833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:14.381762 2833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:54:14.381814 2833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:54:14.381860 2833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:54:14.382866 2833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:14.382962 2833 leader_election.cc:302] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0e320a4526f24e91845b15e1f8e42f1a; no voters:
I20260806 03:54:14.383128 2833 leader_election.cc:288] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 03:54:14.383179 2839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:54:14.383292 2839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [term 3 LEADER]: Becoming Leader. State: Replica: 0e320a4526f24e91845b15e1f8e42f1a, State: Running, Role: LEADER
I20260806 03:54:14.383399 2833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:14.383381 2839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } }
I20260806 03:54:14.383687 2840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "0e320a4526f24e91845b15e1f8e42f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } } }
I20260806 03:54:14.383728 2839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0e320a4526f24e91845b15e1f8e42f1a. Latest consensus state: current_term: 3 leader_uuid: "0e320a4526f24e91845b15e1f8e42f1a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0e320a4526f24e91845b15e1f8e42f1a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 38993 } } }
I20260806 03:54:14.383948 2840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:14.383966 2839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0e320a4526f24e91845b15e1f8e42f1a [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:14.384306 2851 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:14.385102 2851 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:54:14.385208 2851 catalog_manager.cc:1259] Loaded cluster ID: dfb9522ffd75459cb200af9dfc58c67c
I20260806 03:54:14.385254 2851 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260806 03:54:14.385548 2851 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f20b2b319ad google::LogMessage::Fail() at ??:0
@ 0x7f20b2b3488a google::LogMessage::SendToLog() at ??:0
@ 0x7f20b2b314de google::LogMessage::Flush() at ??:0
@ 0x7f20b2b32a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f20baa520f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f20baa5b8b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f20b49ff0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f20b49f4954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f20b58586db start_thread at ??:0
@ 0x7f20b162f61f clone at ??:0
W20260806 03:54:14.490988 2836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:14.491195 2836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:14.491218 2836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:14.492422 2836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:14.492468 2836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:14.492493 2836 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:54:14.494174 2836 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:35647
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=33529
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:14.494412 2836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:14.494637 2836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:14.499645 2882 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:14.499740 2884 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:14.499774 2890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:14.499809 2836 server_base.cc:1034] running on GCE node
I20260806 03:54:14.500018 2836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:14.500252 2836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:14.501395 2836 hybrid_clock.cc:648] HybridClock initialized: now 1785988454501382 us; error 30 us; skew 500 ppm
Aug 06 03:54:14 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988454, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:14.503808 2836 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:54:14.504262 2836 webserver.cc:460] Webserver started at http://127.12.15.65:33529/ using document root <none> and password file <none>
I20260806 03:54:14.504479 2836 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:14.504530 2836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:14.505897 2836 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:14.506670 2903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.506832 2836 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.506920 2836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "5268740ff3d342a9a1dc1d7a9dc63c82"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "b84bbe50a75ee2df7e4c7c0a8b7983da"
server_key_iv: "40789113fa0f328a81943f8151e765b1"
server_key_version: "encryptionkey@0"
I20260806 03:54:14.507262 2836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:14.526427 2836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:14.528518 2836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:14.528651 2836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:14.528861 2836 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:14.529289 2912 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 03:54:14.531353 2836 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:54:14.531399 2836 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 03:54:14.531433 2836 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:54:14.532039 2836 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:54:14.532073 2836 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 03:54:14.532181 2912 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Bootstrap starting.
I20260806 03:54:14.534098 2912 log.cc:824] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:14.537129 2912 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:54:14.537432 2912 tablet_bootstrap.cc:492] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Bootstrap complete.
I20260806 03:54:14.537623 2912 ts_tablet_manager.cc:1397] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260806 03:54:14.538808 2836 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:35647
I20260806 03:54:14.539124 2836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:54:14.539520 3074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:35647 every 8 connection(s)
I20260806 03:54:14.539590 2912 raft_consensus.cc:348] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.540081 2912 raft_consensus.cc:729] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5268740ff3d342a9a1dc1d7a9dc63c82, State: Initialized, Role: FOLLOWER
I20260806 03:54:14.540231 2912 consensus_queue.cc:260] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:14.540326 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 2836
I20260806 03:54:14.540505 2912 ts_tablet_manager.cc:1428] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82: Time spent starting tablet: real 0.003s user 0.006s sys 0.000s
I20260806 03:54:14.540584 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:41783
--local_ip_for_outbound_sockets=127.12.15.66
--tserver_master_addrs=127.12.15.126:38993
--webserver_port=39791
--webserver_interface=127.12.15.66
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:14.621075 3079 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:14.621287 3079 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:14.621318 3079 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:14.622463 3079 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:14.622503 3079 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:14.622538 3079 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:54:14.623966 3079 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:41783
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=39791
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3079
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:14.624209 3079 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:14.624414 3079 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:14.633430 3091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:14.633442 3089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:14.633441 3086 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:14.633554 3079 server_base.cc:1034] running on GCE node
I20260806 03:54:14.633973 3079 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:14.634214 3079 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:14.635352 3079 hybrid_clock.cc:648] HybridClock initialized: now 1785988454635338 us; error 32 us; skew 500 ppm
W20260806 03:54:14.636472 3077 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:54:14.539786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:48985)
0806 03:54:14.539955 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.539999 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:14.542196 (+ 2197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:14.636278 (+ 94082us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
W20260806 03:54:14.636684 3075 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:38993 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.126:38993: Client connection negotiation failed: client connection to 127.12.15.126:38993: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:54:14.636968 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.636856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:41155)
0806 03:54:14.636904 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.636944 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 03:54:14.637295 3077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.637214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:39779)
0806 03:54:14.637254 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.637277 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
Aug 06 03:54:14 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988454, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260806 03:54:14.637761 3075 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:54:14.638007 3079 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:54:14.638458 3079 webserver.cc:460] Webserver started at http://127.12.15.66:39791/ using document root <none> and password file <none>
I20260806 03:54:14.638696 3079 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:14.638747 3079 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:14.640013 3079 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:14.640515 3101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.640666 3079 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:14.640748 3079 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "37cc06ea59e37912ff4925cb4b8e9fad"
server_key_iv: "49c1e0a0cce28d3f746aca5fe49e96e1"
server_key_version: "encryptionkey@0"
I20260806 03:54:14.641065 3079 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:14.652981 3079 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:14.655083 3079 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:14.655215 3079 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:14.655416 3079 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:14.655696 3079 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:14.655730 3079 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.655757 3079 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:14.655781 3079 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.661480 3079 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:41783
I20260806 03:54:14.661536 3270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:41783 every 8 connection(s)
I20260806 03:54:14.661854 3079 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:54:14.662860 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.662483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:56819)
0806 03:54:14.662634 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.662720 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":69,"threads_started":1}
W20260806 03:54:14.663223 3273 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:38993 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.126:38993: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
I20260806 03:54:14.663524 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.663442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:50999)
0806 03:54:14.663482 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.663507 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 03:54:14.663756 3275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.663690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:42739)
0806 03:54:14.663724 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.663742 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 03:54:14.663882 3273 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:54:14.666302 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 3079
I20260806 03:54:14.666539 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:37347
--local_ip_for_outbound_sockets=127.12.15.67
--tserver_master_addrs=127.12.15.126:38993
--webserver_port=42835
--webserver_interface=127.12.15.67
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:14.749673 3277 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:14.749868 3277 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:14.749898 3277 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:14.751019 3277 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:14.751060 3277 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:14.751093 3277 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:54:14.752525 3277 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:37347
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=42835
--tserver_master_addrs=127.12.15.126:38993
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3277
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:14.752753 3277 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:14.752964 3277 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:14.757797 3289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:14.757820 3284 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:14.757824 3285 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:14.757844 3277 server_base.cc:1034] running on GCE node
I20260806 03:54:14.758169 3277 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:14.758387 3277 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:14.759544 3277 hybrid_clock.cc:648] HybridClock initialized: now 1785988454759531 us; error 27 us; skew 500 ppm
Aug 06 03:54:14 dist-test-slave-bqcl krb5kdc[1141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988454, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:14.761925 3277 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:54:14.762351 3277 webserver.cc:460] Webserver started at http://127.12.15.67:42835/ using document root <none> and password file <none>
I20260806 03:54:14.762545 3277 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:14.762584 3277 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:14.763749 3277 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:14.764278 3307 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.764425 3277 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.764508 3277 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "516e1c592f424ebd8ec8487398ef7cc1"
format_stamp: "Formatted at 2026-08-06 03:54:13 on dist-test-slave-bqcl"
server_key: "6aae94d1091a762db0f14321f1c0771a"
server_key_iv: "dd42fd4723ba93965157ad3050b6279c"
server_key_version: "encryptionkey@0"
I20260806 03:54:14.764815 3277 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:14.775465 3277 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:14.777091 3277 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:14.777194 3277 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:14.777367 3277 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:14.777617 3277 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:14.777647 3277 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.777667 3277 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:14.777683 3277 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:14.783226 3277 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:37347
I20260806 03:54:14.783298 3466 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:37347 every 8 connection(s)
I20260806 03:54:14.783576 3277 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:54:14.784547 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.784168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:43511)
0806 03:54:14.784330 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.784406 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":88,"threads_started":1}
W20260806 03:54:14.784925 3468 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:38993 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.126:38993: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
I20260806 03:54:14.785180 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.785095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:49685)
0806 03:54:14.785135 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.785160 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 03:54:14.785434 3470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:14.785358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:45557)
0806 03:54:14.785391 (+ 33us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:14.785412 (+ 21us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 03:54:14.785523 3468 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:54:14.792443 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 3277
I20260806 03:54:15.638314 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:15.638067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:55307)
0806 03:54:15.638207 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:15.638243 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260806 03:54:15.664373 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:15.664172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:47681)
0806 03:54:15.664293 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:15.664324 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":70,"threads_started":1}
I20260806 03:54:15.786062 3476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:15.785849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:41615)
0806 03:54:15.785966 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:15.786006 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":59,"threads_started":1}
I20260806 03:54:16.269289 3477 raft_consensus.cc:482] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:16.269418 3477 raft_consensus.cc:504] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:16.269699 3477 leader_election.cc:288] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783), 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347)
Aug 06 03:54:16 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988454, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.67@KRBTEST.COM
Aug 06 03:54:16 dist-test-slave-bqcl krb5kdc[1141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988454, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.66@KRBTEST.COM
I20260806 03:54:16.284301 3483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:16.269998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:37731 (local address 127.12.15.67:37347)
0806 03:54:16.270177 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:16.270184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:16.270232 (+ 48us) server_negotiation.cc:407] Connection header received
0806 03:54:16.270756 (+ 524us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:16.270759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:16.270820 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:16.270971 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:16.272009 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:16.273429 (+ 1420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:16.274996 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:16.275649 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:16.275772 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:16.281047 (+ 5275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:16.281051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:16.281051 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:16.281063 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:16.282732 (+ 1669us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:16.283163 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:16.283166 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:16.283167 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:16.283259 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:16.283481 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:16.283481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:16.283482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:16.283712 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:16.283794 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:16.284070 (+ 276us) server_negotiation.cc:299] Negotiation successful
0806 03:54:16.284187 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":82,"threads_started":1}
I20260806 03:54:16.284309 3479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:16.269836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:41783 (local address 127.12.15.65:54875)
0806 03:54:16.269950 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:16.269962 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:16.270021 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:16.271036 (+ 1015us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:16.271040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:16.271538 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:16.272076 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:16.272078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:16.273548 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:16.273549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:16.274916 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:16.274920 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:16.275738 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:16.275739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:16.275905 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:16.279110 (+ 3205us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:16.279122 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:16.281248 (+ 2126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:16.282881 (+ 1633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:16.282882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:16.282883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:16.283117 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:16.283343 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:16.283343 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:16.283344 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:16.283425 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:16.283788 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:16.283794 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:16.283918 (+ 124us) client_negotiation.cc:769] Sending connection context
0806 03:54:16.284006 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 03:54:16.284196 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101,"mutex_wait_us":80,"thread_start_us":46,"threads_started":1}
I20260806 03:54:16.284389 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:16.269894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:37347 (local address 127.12.15.65:37731)
0806 03:54:16.270005 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:16.270016 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:16.270070 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:16.271036 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:16.271040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:16.271538 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:16.271893 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:16.271903 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:16.273540 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:16.273541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:16.274921 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:16.274922 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:16.275738 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:16.275739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:16.275905 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:16.279110 (+ 3205us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:16.279122 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:16.280930 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:16.282812 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:16.282819 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:16.282821 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:16.283088 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:16.283318 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:16.283319 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:16.283319 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:16.283423 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:16.283786 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:16.283793 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:16.283919 (+ 126us) client_negotiation.cc:769] Sending connection context
0806 03:54:16.284007 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 03:54:16.284278 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"mutex_wait_us":213,"thread_start_us":58,"threads_started":1}
I20260806 03:54:16.284305 3481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:16.269948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:54875 (local address 127.12.15.66:41783)
0806 03:54:16.270143 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:16.270157 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:16.270184 (+ 27us) server_negotiation.cc:407] Connection header received
0806 03:54:16.270756 (+ 572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:16.270759 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:16.270820 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:16.270971 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:16.272169 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:16.273437 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:16.274995 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:16.275649 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:16.275768 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:16.281342 (+ 5574us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:16.281345 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:16.281345 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:16.281356 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:16.282802 (+ 1446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:16.283186 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:16.283190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:16.283190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:16.283278 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:16.283481 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:16.283482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:16.283482 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:16.283723 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:16.283794 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:16.284072 (+ 278us) server_negotiation.cc:299] Negotiation successful
0806 03:54:16.284189 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":56,"threads_started":1}
W20260806 03:54:16.285336 2905 leader_election.cc:341] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347): Not found: Tablet not found: c8b478be25c54200ae5672255278685a
W20260806 03:54:16.285434 2908 leader_election.cc:341] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783): Not found: Tablet not found: c8b478be25c54200ae5672255278685a
I20260806 03:54:16.285493 2908 leader_election.cc:302] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5268740ff3d342a9a1dc1d7a9dc63c82; no voters: 516e1c592f424ebd8ec8487398ef7cc1, e6351d2ff6f74e1e82fb88fbe6b5bc2c
I20260806 03:54:16.285617 3477 raft_consensus.cc:2726] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:54:16.638821 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:16.638695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:54521)
0806 03:54:16.638749 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:16.638778 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260806 03:54:16.665001 3485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:16.664783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:37181)
0806 03:54:16.664911 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:16.664955 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260806 03:54:16.786741 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:16.786524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:59537)
0806 03:54:16.786648 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:16.786692 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":59,"threads_started":1}
I20260806 03:54:17.639400 3487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:17.639207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:46195)
0806 03:54:17.639319 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:17.639350 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":63,"threads_started":1}
I20260806 03:54:17.665608 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:17.665421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:57507)
0806 03:54:17.665531 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:17.665564 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":61,"threads_started":1}
I20260806 03:54:17.787403 3490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:17.787196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:44761)
0806 03:54:17.787323 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:17.787360 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260806 03:54:18.029597 3491 raft_consensus.cc:482] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:18.029690 3491 raft_consensus.cc:504] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:18.029826 3491 leader_election.cc:288] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783), 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347)
W20260806 03:54:18.030272 2908 leader_election.cc:341] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783): Not found: Tablet not found: c8b478be25c54200ae5672255278685a
W20260806 03:54:18.030369 2905 leader_election.cc:341] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347): Not found: Tablet not found: c8b478be25c54200ae5672255278685a
I20260806 03:54:18.030421 2905 leader_election.cc:302] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5268740ff3d342a9a1dc1d7a9dc63c82; no voters: 516e1c592f424ebd8ec8487398ef7cc1, e6351d2ff6f74e1e82fb88fbe6b5bc2c
I20260806 03:54:18.030498 3491 raft_consensus.cc:2726] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:54:18.640014 3492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:18.639789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:39779)
0806 03:54:18.639928 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:18.639964 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":86,"threads_started":1}
I20260806 03:54:18.666244 3494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:18.665973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:35267)
0806 03:54:18.666102 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:18.666156 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":61,"threads_started":1}
I20260806 03:54:18.788051 3495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:18.787827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:53907)
0806 03:54:18.787973 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:18.788009 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":81,"threads_started":1}
I20260806 03:54:19.640630 3497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:19.640414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:49845)
0806 03:54:19.640544 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:19.640579 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260806 03:54:19.666847 3498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:19.666650 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:51777)
0806 03:54:19.666767 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:19.666803 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260806 03:54:19.788641 3499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:19.788434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:53597)
0806 03:54:19.788562 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:19.788598 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":60,"threads_started":1}
I20260806 03:54:19.845772 3501 raft_consensus.cc:482] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:19.845878 3501 raft_consensus.cc:504] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:19.846161 3501 leader_election.cc:288] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783), 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347)
W20260806 03:54:19.846377 2908 leader_election.cc:341] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783): Not found: Tablet not found: c8b478be25c54200ae5672255278685a
W20260806 03:54:19.846447 2905 leader_election.cc:341] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347): Not found: Tablet not found: c8b478be25c54200ae5672255278685a
I20260806 03:54:19.846489 2905 leader_election.cc:302] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5268740ff3d342a9a1dc1d7a9dc63c82; no voters: 516e1c592f424ebd8ec8487398ef7cc1, e6351d2ff6f74e1e82fb88fbe6b5bc2c
I20260806 03:54:19.846571 3501 raft_consensus.cc:2726] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:54:20.641256 3502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:20.641025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:55687)
0806 03:54:20.641164 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:20.641203 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":64,"threads_started":1}
I20260806 03:54:20.667461 3504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:20.667267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:41797)
0806 03:54:20.667379 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:20.667419 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":60,"threads_started":1}
I20260806 03:54:20.789322 3505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:20.789069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:42953)
0806 03:54:20.789224 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:20.789273 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":76,"threads_started":1}
I20260806 03:54:21.641945 3506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:21.641671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:48569)
0806 03:54:21.641853 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:21.641893 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":104,"threads_started":1}
I20260806 03:54:21.668056 3507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:21.667853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:35125)
0806 03:54:21.667987 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:21.668017 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
I20260806 03:54:21.789963 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:21.789750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:47699)
0806 03:54:21.789884 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:21.789920 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":79,"threads_started":1}
I20260806 03:54:22.642601 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:22.642373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:56713)
0806 03:54:22.642506 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:22.642544 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":81,"threads_started":1}
I20260806 03:54:22.668663 3511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:22.668456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:33071)
0806 03:54:22.668588 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:22.668619 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":78,"threads_started":1}
I20260806 03:54:22.790611 3513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:22.790404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:34725)
0806 03:54:22.790533 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:22.790567 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":67,"threads_started":1}
I20260806 03:54:23.643275 3516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:23.643020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:57439)
0806 03:54:23.643174 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:23.643224 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":71,"threads_started":1}
I20260806 03:54:23.669279 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:23.669059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:47309)
0806 03:54:23.669200 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:23.669234 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":77,"threads_started":1}
I20260806 03:54:23.791258 3518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:23.791041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:60899)
0806 03:54:23.791170 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:23.791210 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":66,"threads_started":1}
I20260806 03:54:24.332309 3519 raft_consensus.cc:482] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:24.332418 3519 raft_consensus.cc:504] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e6351d2ff6f74e1e82fb88fbe6b5bc2c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41783 } } peers { permanent_uuid: "516e1c592f424ebd8ec8487398ef7cc1" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 37347 } } peers { permanent_uuid: "5268740ff3d342a9a1dc1d7a9dc63c82" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 35647 } }
I20260806 03:54:24.332590 3519 leader_election.cc:288] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783), 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347)
W20260806 03:54:24.332989 2905 leader_election.cc:341] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 516e1c592f424ebd8ec8487398ef7cc1 (127.12.15.67:37347): Not found: Tablet not found: c8b478be25c54200ae5672255278685a
W20260806 03:54:24.333081 2908 leader_election.cc:341] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e6351d2ff6f74e1e82fb88fbe6b5bc2c (127.12.15.66:41783): Not found: Tablet not found: c8b478be25c54200ae5672255278685a
I20260806 03:54:24.333141 2908 leader_election.cc:302] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5268740ff3d342a9a1dc1d7a9dc63c82; no voters: 516e1c592f424ebd8ec8487398ef7cc1, e6351d2ff6f74e1e82fb88fbe6b5bc2c
I20260806 03:54:24.333216 3519 raft_consensus.cc:2726] T c8b478be25c54200ae5672255278685a P 5268740ff3d342a9a1dc1d7a9dc63c82 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:54:24.644011 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:24.643767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:33183)
0806 03:54:24.643903 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:24.643960 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260806 03:54:24.669911 3521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:24.669675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:46955)
0806 03:54:24.669808 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:24.669849 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":61,"threads_started":1}
I20260806 03:54:24.791913 3522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:24.791684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:33791)
0806 03:54:24.791824 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:24.791863 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":68,"threads_started":1}
I20260806 03:54:25.644694 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:25.644411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:47543)
0806 03:54:25.644597 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:25.644641 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":64,"threads_started":1}
I20260806 03:54:25.670550 3524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:25.670336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:49719)
0806 03:54:25.670474 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:25.670505 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":80,"threads_started":1}
I20260806 03:54:25.792574 3526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:25.792343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:54057)
0806 03:54:25.792478 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:25.792526 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":80,"threads_started":1}
I20260806 03:54:26.645396 3527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:26.645121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:43865)
0806 03:54:26.645298 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:26.645344 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":94,"threads_started":1}
I20260806 03:54:26.671193 3528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:26.670967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:39793)
0806 03:54:26.671112 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:26.671144 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":81,"threads_started":1}
I20260806 03:54:26.793211 3530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:26.792977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:57929)
0806 03:54:26.793125 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:26.793163 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":85,"threads_started":1}
I20260806 03:54:27.646013 3532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:27.645815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:46073)
0806 03:54:27.645927 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:27.645962 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":65,"threads_started":1}
I20260806 03:54:27.671833 3533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:27.671598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:33241)
0806 03:54:27.671740 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:27.671775 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":73,"threads_started":1}
I20260806 03:54:27.793881 3534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:27.793618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:51125)
0806 03:54:27.793755 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:27.793834 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260806 03:54:28.646752 3535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:28.646486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:32885)
0806 03:54:28.646644 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:28.646687 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":78,"threads_started":1}
I20260806 03:54:28.672833 3536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:28.672362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:44783)
0806 03:54:28.672643 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:28.672719 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":181,"threads_started":1}
I20260806 03:54:28.794591 3537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:28.794339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:33179)
0806 03:54:28.794481 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:28.794542 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":63,"threads_started":1}
I20260806 03:54:29.647431 3538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:29.647227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.65:48389)
0806 03:54:29.647352 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:29.647383 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":62,"threads_started":1}
I20260806 03:54:29.673621 3543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:29.673378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.66:40567)
0806 03:54:29.673531 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:29.673573 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260806 03:54:29.795303 3545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:29.795059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:38993 (local address 127.12.15.67:55685)
0806 03:54:29.795224 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:29.795256 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:38993: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":85,"threads_started":1}
I20260806 03:54:29.802723 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 2836
I20260806 03:54:29.815546 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 3079
I20260806 03:54:29.834036 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 3277
2026-08-06T03:54:29Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18349 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:54:29 dist-test-slave-bqcl krb5kdc[3552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:54:29 dist-test-slave-bqcl krb5kdc[3552](info): set up 2 sockets
Aug 06 03:54:29 dist-test-slave-bqcl krb5kdc[3552](info): commencing operation
krb5kdc: starting...
W20260806 03:54:30.994477 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:54:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:54:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:54:31.112196 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33579
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:31.193353 3576 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:31.193560 3576 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:31.193591 3576 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:31.194738 3576 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 03:54:31.194777 3576 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 03:54:31.194793 3576 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 03:54:31.194808 3576 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:31.194823 3576 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:31.194839 3576 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:31.194855 3576 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:31.194878 3576 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:31.196432 3576 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:33579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3576
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:31.196683 3576 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:31.196884 3576 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:31.201752 3583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:31.201678 3581 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:31.201716 3576 server_base.cc:1034] running on GCE node
W20260806 03:54:31.201681 3588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:31.202275 3576 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:31.202539 3576 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:31.203683 3576 hybrid_clock.cc:648] HybridClock initialized: now 1785988471203670 us; error 31 us; skew 500 ppm
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:31.206103 3576 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:31.206619 3576 webserver.cc:460] Webserver started at http://127.12.15.126:43649/ using document root <none> and password file <none>
I20260806 03:54:31.206825 3576 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:31.206889 3576 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:31.207031 3576 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:31.208117 3576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "dd1b66b4b2664aea9b3ba37186861db4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "6743d71a0c3a202a303eb977fde7a409"
server_key_iv: "83ab38f99913aa6dd7b554ab63030367"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.208482 3576 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "dd1b66b4b2664aea9b3ba37186861db4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "6743d71a0c3a202a303eb977fde7a409"
server_key_iv: "83ab38f99913aa6dd7b554ab63030367"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.209800 3576 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:54:31.210582 3602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.210770 3576 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.210863 3576 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "dd1b66b4b2664aea9b3ba37186861db4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "6743d71a0c3a202a303eb977fde7a409"
server_key_iv: "83ab38f99913aa6dd7b554ab63030367"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.210924 3576 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:31.242076 3576 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:31.243871 3576 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:31.244016 3576 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:31.248064 3576 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:33579
I20260806 03:54:31.248106 3668 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:33579 every 8 connection(s)
I20260806 03:54:31.248445 3576 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:31.249315 3669 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:31.251647 3669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Bootstrap starting.
I20260806 03:54:31.252205 3669 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:31.252447 3669 log.cc:824] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:31.253080 3669 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: No bootstrap required, opened a new log
I20260806 03:54:31.254633 3669 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:54:31.254788 3669 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:31.254810 3669 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1b66b4b2664aea9b3ba37186861db4, State: Initialized, Role: FOLLOWER
I20260806 03:54:31.254925 3669 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:54:31.255002 3669 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:54:31.255029 3669 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:54:31.255062 3669 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:31.255537 3669 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:54:31.255623 3669 leader_election.cc:302] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd1b66b4b2664aea9b3ba37186861db4; no voters:
I20260806 03:54:31.255751 3669 leader_election.cc:288] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:54:31.255807 3672 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:31.255964 3672 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 1 LEADER]: Becoming Leader. State: Replica: dd1b66b4b2664aea9b3ba37186861db4, State: Running, Role: LEADER
I20260806 03:54:31.255987 3669 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:31.256058 3672 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:54:31.256460 3675 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd1b66b4b2664aea9b3ba37186861db4. Latest consensus state: current_term: 1 leader_uuid: "dd1b66b4b2664aea9b3ba37186861db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } } }
I20260806 03:54:31.256441 3673 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd1b66b4b2664aea9b3ba37186861db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } } }
I20260806 03:54:31.256561 3675 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:31.256567 3673 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:31.256784 3682 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:31.257330 3682 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:54:31.257371 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 3576
I20260806 03:54:31.257473 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:54:31.257683 12349 external_mini_cluster.cc:1428] Setting key 4d69fd3026100a001a14935dd7cd8e23
I20260806 03:54:31.258900 3682 catalog_manager.cc:1347] Generated new cluster ID: 0d6868d3b96f4aefbd68f826e14f623c
I20260806 03:54:31.258988 3682 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:54:31.264041 3682 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:54:31.264725 3682 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:31.268039 3682 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Generated new TSK 0
I20260806 03:54:31.268210 3682 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:54:31.269343 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.258173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.12.15.126:33579)
0806 03:54:31.258621 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.258630 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.258661 (+ 31us) server_negotiation.cc:407] Connection header received
0806 03:54:31.258776 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.258783 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.258838 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.258968 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:31.259721 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.261139 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.262168 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.262800 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.262910 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.265853 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:31.265857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:31.265857 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:31.265870 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:31.267849 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.268243 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.268246 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.268246 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.268358 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.268555 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.268556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.268556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.268767 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:31.268912 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.269007 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.269166 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":396,"thread_start_us":75,"threads_started":1}
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:54:31.293967 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33579
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:31.374902 3711 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:31.375098 3711 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:31.375120 3711 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:31.376232 3711 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:31.376272 3711 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:31.376297 3711 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:54:31.377712 3711 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3711
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:31.377923 3711 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:31.378156 3711 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:31.382974 3720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:31.382972 3721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:31.382972 3725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:31.383173 3711 server_base.cc:1034] running on GCE node
I20260806 03:54:31.383421 3711 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:31.383646 3711 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:31.384793 3711 hybrid_clock.cc:648] HybridClock initialized: now 1785988471384782 us; error 29 us; skew 500 ppm
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:31.387324 3711 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:54:31.387718 3711 webserver.cc:460] Webserver started at http://127.12.15.65:42861/ using document root <none> and password file <none>
I20260806 03:54:31.387907 3711 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:31.387953 3711 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:31.388083 3711 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:31.389091 3711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "e50e26fef5d745de87690e08ad830ea9"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "a2b53233b9f2c00296657b61ca5041d8"
server_key_iv: "8f35c38ffaceea6052a7c179592a981f"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.389417 3711 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "e50e26fef5d745de87690e08ad830ea9"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "a2b53233b9f2c00296657b61ca5041d8"
server_key_iv: "8f35c38ffaceea6052a7c179592a981f"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.390676 3711 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:54:31.391304 3737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.391453 3711 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:31.391534 3711 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "e50e26fef5d745de87690e08ad830ea9"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "a2b53233b9f2c00296657b61ca5041d8"
server_key_iv: "8f35c38ffaceea6052a7c179592a981f"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.391598 3711 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:31.400697 3711 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:31.402324 3711 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:31.402451 3711 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:31.402662 3711 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:31.402951 3711 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:31.402988 3711 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.403016 3711 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:31.403033 3711 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.408609 3711 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:40919
I20260806 03:54:31.408674 3903 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:40919 every 8 connection(s)
I20260806 03:54:31.408954 3711 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:31.417470 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.409669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:33313 (local address 127.12.15.126:33579)
0806 03:54:31.409736 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.409738 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.409926 (+ 188us) server_negotiation.cc:407] Connection header received
0806 03:54:31.410499 (+ 573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.410500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.410510 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.410560 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:31.411450 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.411942 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.412895 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.413089 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.413136 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.415224 (+ 2088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:31.415225 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:31.415226 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:31.415232 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:31.416475 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.416797 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.416798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.416798 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.416847 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.417017 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.417017 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.417018 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.417111 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:31.417155 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.417343 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.417405 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:54:31.417629 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.409656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:33313)
0806 03:54:31.409812 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.409847 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:31.410351 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:31.410619 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:31.410622 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:31.411176 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:31.411380 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.411386 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.412001 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.412002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.412828 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.412831 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.413143 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.413144 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.413248 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.413733 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:31.413741 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:31.415124 (+ 1383us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:31.416538 (+ 1414us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:31.416542 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:31.416543 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:31.416742 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:31.416890 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:31.416890 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:31.416890 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:31.416977 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:31.417151 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:31.417154 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:31.417256 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 03:54:31.417329 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:54:31.417439 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
I20260806 03:54:31.418053 3904 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33579
I20260806 03:54:31.418162 3904 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:31.418335 3904 heartbeater.cc:511] Master 127.12.15.126:33579 requested a full tablet report, sending...
I20260806 03:54:31.418846 3624 ts_manager.cc:194] Registered new tserver with Master: e50e26fef5d745de87690e08ad830ea9 (127.12.15.65:40919)
I20260806 03:54:31.418919 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 3711
I20260806 03:54:31.419028 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:54:31.419205 12349 external_mini_cluster.cc:1428] Setting key 889f181993d8ea28bc4f514be07a6bf2
I20260806 03:54:31.419749 3624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:33313
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:54:31.442991 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33579
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:31.523165 3912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:31.523367 3912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:31.523389 3912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:31.524444 3912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:31.524477 3912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:31.524500 3912 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:54:31.525875 3912 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:31.526054 3912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:31.526250 3912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:31.530858 3922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:31.530866 3924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:31.530866 3928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:31.530929 3912 server_base.cc:1034] running on GCE node
I20260806 03:54:31.531199 3912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:31.531411 3912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:31.532547 3912 hybrid_clock.cc:648] HybridClock initialized: now 1785988471532529 us; error 31 us; skew 500 ppm
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:31.535549 3912 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:54:31.536031 3912 webserver.cc:460] Webserver started at http://127.12.15.66:39913/ using document root <none> and password file <none>
I20260806 03:54:31.536280 3912 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:31.536327 3912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:31.536491 3912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:31.537458 3912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "7f926da79c511b9d495c52f481302704"
server_key_iv: "905704251723c8776d03a2859f98edcc"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.537742 3912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "7f926da79c511b9d495c52f481302704"
server_key_iv: "905704251723c8776d03a2859f98edcc"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.538976 3912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:54:31.539642 3938 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.539906 3912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:31.539988 3912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "7f926da79c511b9d495c52f481302704"
server_key_iv: "905704251723c8776d03a2859f98edcc"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.540040 3912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:31.555325 3912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:31.557337 3912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:31.557446 3912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:31.557638 3912 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:31.557968 3912 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:31.558003 3912 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.558022 3912 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:31.558045 3912 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.563666 3912 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:44505
I20260806 03:54:31.563742 4098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:44505 every 8 connection(s)
I20260806 03:54:31.563995 3912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:54:31.567745 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 3912
I20260806 03:54:31.567842 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:54:31.568033 12349 external_mini_cluster.cc:1428] Setting key 55b8478db67b31b7637678deab1a0d2e
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:31.572587 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.564601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:39285 (local address 127.12.15.126:33579)
0806 03:54:31.564674 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.564676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.564822 (+ 146us) server_negotiation.cc:407] Connection header received
0806 03:54:31.565253 (+ 431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.565254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.565264 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.565310 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:31.566173 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.566661 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.567588 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.567772 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.567823 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.569891 (+ 2068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:31.569893 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:31.569894 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:31.569901 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:31.571152 (+ 1251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.571628 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.571629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.571629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.571700 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.571978 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.571979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.571979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.572095 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:31.572154 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.572441 (+ 287us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.572519 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260806 03:54:31.572805 4104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.564572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:39285)
0806 03:54:31.564713 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.564745 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:31.565170 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:31.565363 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:31.565366 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:31.565897 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:31.566078 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.566083 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.566747 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.566748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.567527 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.567529 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.567851 (+ 322us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.567852 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.567991 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.568438 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:31.568446 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:31.569805 (+ 1359us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:31.571231 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:31.571239 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:31.571241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:31.571530 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:31.571765 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:31.571766 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:31.571767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:31.571911 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:31.572156 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:31.572162 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:31.572315 (+ 153us) client_negotiation.cc:769] Sending connection context
0806 03:54:31.572425 (+ 110us) client_negotiation.cc:240] Negotiation successful
0806 03:54:31.572580 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":80,"threads_started":1}
I20260806 03:54:31.573333 4100 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33579
I20260806 03:54:31.573444 4100 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:31.573678 4100 heartbeater.cc:511] Master 127.12.15.126:33579 requested a full tablet report, sending...
I20260806 03:54:31.574059 3624 ts_manager.cc:194] Registered new tserver with Master: 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505)
I20260806 03:54:31.574442 3624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:39285
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:54:31.594220 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33579
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:31.674583 4111 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:31.674789 4111 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:31.674819 4111 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:31.675928 4111 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:31.675968 4111 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:31.676002 4111 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:54:31.677407 4111 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4111
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:31.677632 4111 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:31.677851 4111 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:31.682628 4118 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:31.682675 4111 server_base.cc:1034] running on GCE node
W20260806 03:54:31.682641 4119 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:31.682642 4123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:31.683038 4111 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:31.683264 4111 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:31.684403 4111 hybrid_clock.cc:648] HybridClock initialized: now 1785988471684391 us; error 29 us; skew 500 ppm
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:31.686771 4111 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:54:31.687179 4111 webserver.cc:460] Webserver started at http://127.12.15.67:45635/ using document root <none> and password file <none>
I20260806 03:54:31.687366 4111 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:31.687414 4111 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:31.687539 4111 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:54:31.688567 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "e4fc80fa33324e25b48b5a14e953001c"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "79a60ca7922433b4c9c7477685787319"
server_key_iv: "894cfc7d0ed6a4db480e0d3900e22556"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.688887 4111 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "e4fc80fa33324e25b48b5a14e953001c"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "79a60ca7922433b4c9c7477685787319"
server_key_iv: "894cfc7d0ed6a4db480e0d3900e22556"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.690196 4111 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:54:31.690894 4133 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.691094 4111 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:31.691196 4111 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "e4fc80fa33324e25b48b5a14e953001c"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "79a60ca7922433b4c9c7477685787319"
server_key_iv: "894cfc7d0ed6a4db480e0d3900e22556"
server_key_version: "encryptionkey@0"
I20260806 03:54:31.691267 4111 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:31.697966 4111 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:31.699738 4111 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:31.699878 4111 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:31.700086 4111 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:31.700371 4111 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:31.700408 4111 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.700439 4111 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:31.700467 4111 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:31.706588 4111 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:46289
I20260806 03:54:31.706622 4315 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:46289 every 8 connection(s)
I20260806 03:54:31.706926 4111 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:54:31.708731 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 4111
I20260806 03:54:31.708833 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:54:31.709015 12349 external_mini_cluster.cc:1428] Setting key 538c268db80e199ee3ed6d5caf525933
Aug 06 03:54:31 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988471, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:31.715572 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.707630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:49553 (local address 127.12.15.126:33579)
0806 03:54:31.707713 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.707716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.707881 (+ 165us) server_negotiation.cc:407] Connection header received
0806 03:54:31.708356 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.708357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.708369 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.708427 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:31.709439 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.710118 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.711140 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.711331 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.711376 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.713323 (+ 1947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:31.713324 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:31.713324 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:31.713331 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:31.714543 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.714892 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.714893 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.714893 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.714941 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.715112 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.715112 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.715113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.715205 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:31.715251 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.715442 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.715499 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260806 03:54:31.715739 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.707606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:49553)
0806 03:54:31.707765 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.707800 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:31.708251 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:31.708496 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:31.708499 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:31.709098 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:31.709335 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.709343 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.710237 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.710238 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.711069 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.711072 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.711383 (+ 311us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.711384 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.711477 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.711939 (+ 462us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:31.711947 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:31.713252 (+ 1305us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:31.714617 (+ 1365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:31.714623 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:31.714625 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:31.714838 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:31.714986 (+ 148us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:31.714986 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:31.714986 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:31.715072 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:31.715246 (+ 174us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:31.715248 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:31.715351 (+ 103us) client_negotiation.cc:769] Sending connection context
0806 03:54:31.715424 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:54:31.715535 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":85,"threads_started":1}
I20260806 03:54:31.716151 4316 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33579
I20260806 03:54:31.716236 4316 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:31.716403 4316 heartbeater.cc:511] Master 127.12.15.126:33579 requested a full tablet report, sending...
I20260806 03:54:31.716740 3624 ts_manager.cc:194] Registered new tserver with Master: e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
I20260806 03:54:31.717089 3624 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:49553
I20260806 03:54:31.720577 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:54:31.726948 3703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.721857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38408 (local address 127.12.15.126:33579)
0806 03:54:31.721913 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.721915 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.722017 (+ 102us) server_negotiation.cc:407] Connection header received
0806 03:54:31.722058 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.722058 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.722065 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.722109 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:31.722624 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.723063 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.723717 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.723879 (+ 162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.723919 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.724968 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:31.724969 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:31.724969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:31.724975 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:31.726041 (+ 1066us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.726363 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.726363 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.726364 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.726413 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:31.726542 (+ 129us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:31.726542 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:31.726543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:31.726688 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:31.726759 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.726831 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.726887 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:54:31.728399 3624 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38408:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:54:31.729933 3624 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:54:31.736032 4331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.731944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40919 (local address 127.0.0.1:43834)
0806 03:54:31.732414 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.732423 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:31.732481 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:31.732740 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:31.732741 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:31.732743 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:31.732799 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.732801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.733692 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.733692 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.735064 (+ 1372us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.735067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.735743 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.735822 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.735835 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 03:54:31.735899 (+ 64us) client_negotiation.cc:240] Negotiation successful
0806 03:54:31.735954 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":50,"thread_start_us":57,"threads_started":1}
I20260806 03:54:31.736140 4330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.731944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:44505 (local address 127.0.0.1:39934)
0806 03:54:31.732102 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.732140 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:31.732219 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:31.732437 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:31.732440 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:31.732442 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:31.732507 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.732509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.733759 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.733760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.735065 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.735067 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.735887 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.735939 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.735942 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:54:31.735988 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:54:31.736018 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":38,"thread_start_us":52,"threads_started":1}
I20260806 03:54:31.736174 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.731951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:46289 (local address 127.0.0.1:42366)
0806 03:54:31.732529 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.732539 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:31.732591 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:31.732857 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:31.732858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:31.732860 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:31.732889 (+ 29us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.732891 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.734019 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.734020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.734948 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.734951 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.735906 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735907 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.735970 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.735974 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:54:31.736034 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:54:31.736073 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":101,"threads_started":1}
I20260806 03:54:31.736362 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.732178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42366 (local address 127.12.15.67:46289)
0806 03:54:31.732367 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.732373 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.732675 (+ 302us) server_negotiation.cc:407] Connection header received
0806 03:54:31.732716 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.732719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.732734 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.732797 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:31.732953 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.733926 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735020 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.735739 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735810 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.735868 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.736254 (+ 386us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.736295 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":92,"threads_started":1}
I20260806 03:54:31.736366 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.732037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39934 (local address 127.12.15.66:44505)
0806 03:54:31.732241 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.732247 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.732282 (+ 35us) server_negotiation.cc:407] Connection header received
0806 03:54:31.732323 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.732326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.732342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.732421 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:31.732572 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.733658 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735144 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.735818 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735876 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.735923 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.736000 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.736036 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":94,"threads_started":1}
I20260806 03:54:31.736367 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.732186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43834 (local address 127.12.15.65:40919)
0806 03:54:31.732375 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.732379 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.732502 (+ 123us) server_negotiation.cc:407] Connection header received
0806 03:54:31.732552 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.732556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.732571 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.732653 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:31.732873 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.733600 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735171 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.735663 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.735731 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.735792 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.736230 (+ 438us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.736287 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":90,"threads_started":1}
I20260806 03:54:31.737142 3813 tablet_service.cc:1467] Processing CreateTablet for tablet 1df7d5d9b054438aa96d38b384908018 (DEFAULT_TABLE table=test-table [id=4e97ec1893974006ae13edf5463581ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:31.737139 4005 tablet_service.cc:1467] Processing CreateTablet for tablet 1df7d5d9b054438aa96d38b384908018 (DEFAULT_TABLE table=test-table [id=4e97ec1893974006ae13edf5463581ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:31.737501 4005 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1df7d5d9b054438aa96d38b384908018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:31.737501 3813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1df7d5d9b054438aa96d38b384908018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:31.738596 4208 tablet_service.cc:1467] Processing CreateTablet for tablet 1df7d5d9b054438aa96d38b384908018 (DEFAULT_TABLE table=test-table [id=4e97ec1893974006ae13edf5463581ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:31.738889 4208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1df7d5d9b054438aa96d38b384908018. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:31.741030 4340 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Bootstrap starting.
I20260806 03:54:31.741374 4342 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Bootstrap starting.
I20260806 03:54:31.741636 4344 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: Bootstrap starting.
I20260806 03:54:31.741742 4340 tablet_bootstrap.cc:654] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:31.742075 4342 tablet_bootstrap.cc:654] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:31.742283 4340 log.cc:824] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:31.742416 4344 tablet_bootstrap.cc:654] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:31.742735 4342 log.cc:824] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:31.742946 4344 log.cc:824] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:31.743551 4342 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: No bootstrap required, opened a new log
I20260806 03:54:31.743638 4342 ts_tablet_manager.cc:1397] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:31.743659 4340 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: No bootstrap required, opened a new log
I20260806 03:54:31.743710 4340 ts_tablet_manager.cc:1397] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260806 03:54:31.745590 4340 raft_consensus.cc:348] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.745600 4342 raft_consensus.cc:348] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.745779 4340 raft_consensus.cc:374] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:31.745779 4342 raft_consensus.cc:374] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:31.745824 4340 raft_consensus.cc:729] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e98a89ba4bf4a0da198f39bbe34a3c4, State: Initialized, Role: FOLLOWER
I20260806 03:54:31.745824 4342 raft_consensus.cc:729] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e50e26fef5d745de87690e08ad830ea9, State: Initialized, Role: FOLLOWER
I20260806 03:54:31.745944 4340 consensus_queue.cc:260] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.745944 4342 consensus_queue.cc:260] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.746254 4342 ts_tablet_manager.cc:1428] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:54:31.746254 4340 ts_tablet_manager.cc:1428] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260806 03:54:31.746270 4100 heartbeater.cc:503] Master 127.12.15.126:33579 was elected leader, sending a full tablet report...
I20260806 03:54:31.746373 3904 heartbeater.cc:503] Master 127.12.15.126:33579 was elected leader, sending a full tablet report...
I20260806 03:54:31.746855 4344 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: No bootstrap required, opened a new log
I20260806 03:54:31.746915 4344 ts_tablet_manager.cc:1397] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260806 03:54:31.748426 4344 raft_consensus.cc:348] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.748550 4344 raft_consensus.cc:374] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:31.748569 4344 raft_consensus.cc:729] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4fc80fa33324e25b48b5a14e953001c, State: Initialized, Role: FOLLOWER
I20260806 03:54:31.748658 4344 consensus_queue.cc:260] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.748880 4344 ts_tablet_manager.cc:1428] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:31.748927 4316 heartbeater.cc:503] Master 127.12.15.126:33579 was elected leader, sending a full tablet report...
W20260806 03:54:31.814512 4103 tablet.cc:2369] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:54:31.869267 4352 raft_consensus.cc:482] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:31.869411 4352 raft_consensus.cc:504] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.869750 4352 leader_election.cc:288] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e50e26fef5d745de87690e08ad830ea9 (127.12.15.65:40919)
I20260806 03:54:31.872449 4334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.870033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:58361 (local address 127.12.15.66:44505)
0806 03:54:31.870094 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.870096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.870105 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:54:31.870153 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.870154 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.870164 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.870211 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:31.870378 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.870841 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.871742 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.872124 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.872190 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.872200 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.872344 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.872382 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:54:31.872449 4320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.869933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:44505 (local address 127.12.15.67:58361)
0806 03:54:31.870006 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.870018 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:31.870069 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:31.870225 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:31.870226 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:31.870228 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:31.870302 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.870304 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.870917 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.870918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.871654 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.871656 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.872200 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.872201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.872279 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.872282 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:54:31.872328 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:54:31.872370 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260806 03:54:31.872617 4339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.870053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:54141 (local address 127.12.15.65:40919)
0806 03:54:31.870102 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.870103 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.870233 (+ 130us) server_negotiation.cc:407] Connection header received
0806 03:54:31.870291 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.870292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.870303 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.870344 (+ 41us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:31.870490 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.870977 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.871930 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.872302 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.872366 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.872375 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.872512 (+ 137us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.872549 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:54:31.872632 4353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.869959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40919 (local address 127.12.15.67:54141)
0806 03:54:31.870165 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.870173 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:31.870230 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:31.870356 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:31.870356 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:31.870358 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:31.870428 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.870429 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.871055 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.871056 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.871841 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:31.871843 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.872380 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.872381 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:31.872449 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.872452 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:54:31.872498 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:54:31.872545 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":22,"thread_start_us":101,"threads_started":1}
I20260806 03:54:31.872814 4033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1df7d5d9b054438aa96d38b384908018" candidate_uuid: "e4fc80fa33324e25b48b5a14e953001c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" is_pre_election: true
I20260806 03:54:31.872978 4033 raft_consensus.cc:2454] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4fc80fa33324e25b48b5a14e953001c in term 0.
I20260806 03:54:31.872956 3843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1df7d5d9b054438aa96d38b384908018" candidate_uuid: "e4fc80fa33324e25b48b5a14e953001c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e50e26fef5d745de87690e08ad830ea9" is_pre_election: true
I20260806 03:54:31.873101 3843 raft_consensus.cc:2454] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e4fc80fa33324e25b48b5a14e953001c in term 0.
I20260806 03:54:31.873219 4134 leader_election.cc:302] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e98a89ba4bf4a0da198f39bbe34a3c4, e4fc80fa33324e25b48b5a14e953001c; no voters:
I20260806 03:54:31.873358 4352 raft_consensus.cc:2781] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:54:31.873418 4352 raft_consensus.cc:482] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:54:31.873435 4352 raft_consensus.cc:3037] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:31.874166 4352 raft_consensus.cc:504] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.874285 4352 leader_election.cc:288] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [CANDIDATE]: Term 1 election: Requested vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e50e26fef5d745de87690e08ad830ea9 (127.12.15.65:40919)
I20260806 03:54:31.874455 3843 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1df7d5d9b054438aa96d38b384908018" candidate_uuid: "e4fc80fa33324e25b48b5a14e953001c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e50e26fef5d745de87690e08ad830ea9"
I20260806 03:54:31.874447 4033 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1df7d5d9b054438aa96d38b384908018" candidate_uuid: "e4fc80fa33324e25b48b5a14e953001c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4"
I20260806 03:54:31.874534 3843 raft_consensus.cc:3037] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:31.874536 4033 raft_consensus.cc:3037] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:31.875232 3843 raft_consensus.cc:2454] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4fc80fa33324e25b48b5a14e953001c in term 1.
I20260806 03:54:31.875239 4033 raft_consensus.cc:2454] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e4fc80fa33324e25b48b5a14e953001c in term 1.
I20260806 03:54:31.875420 4136 leader_election.cc:302] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4fc80fa33324e25b48b5a14e953001c, e50e26fef5d745de87690e08ad830ea9; no voters:
I20260806 03:54:31.875509 4352 raft_consensus.cc:2781] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:31.875664 4352 raft_consensus.cc:686] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 1 LEADER]: Becoming Leader. State: Replica: e4fc80fa33324e25b48b5a14e953001c, State: Running, Role: LEADER
I20260806 03:54:31.875744 4352 consensus_queue.cc:237] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:31.876593 3623 catalog_manager.cc:5495] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c reported cstate change: term changed from 0 to 1, leader changed from <none> to e4fc80fa33324e25b48b5a14e953001c (127.12.15.67). New cstate: current_term: 1 leader_uuid: "e4fc80fa33324e25b48b5a14e953001c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } health_report { overall_health: UNKNOWN } } }
I20260806 03:54:31.882902 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.880765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42382 (local address 127.12.15.67:46289)
0806 03:54:31.880805 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:31.880807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:31.880832 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:54:31.880898 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:31.880898 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:31.880915 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:31.880962 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:54:31.881053 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.881476 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.882198 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:31.882380 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:31.882442 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:31.882632 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:54:31.882733 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:54:31.882798 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:31.882812 (+ 14us) server_negotiation.cc:299] Negotiation successful
0806 03:54:31.882828 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:54:31.884464 3843 raft_consensus.cc:1261] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Refusing update from remote peer e4fc80fa33324e25b48b5a14e953001c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:31.884503 4033 raft_consensus.cc:1261] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 1 FOLLOWER]: Refusing update from remote peer e4fc80fa33324e25b48b5a14e953001c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:31.884831 4356 consensus_queue.cc:1035] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:31.884958 4352 consensus_queue.cc:1035] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:31.887293 4360 mvcc.cc:205] Tried to move back new op lower bound from 7315408780836556800 to 7315408780803706880. Current Snapshot: MvccSnapshot[applied={T|T < 7315408780836556800}]
I20260806 03:54:31.887434 4361 mvcc.cc:205] Tried to move back new op lower bound from 7315408780836556800 to 7315408780803706880. Current Snapshot: MvccSnapshot[applied={T|T < 7315408780836556800}]
I20260806 03:54:31.889348 3623 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:31.889444 3623 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38408:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:31.890146 3623 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Sending DeleteTablet for 3 replicas of tablet 1df7d5d9b054438aa96d38b384908018
I20260806 03:54:31.890450 4208 tablet_service.cc:1514] Processing DeleteTablet for tablet 1df7d5d9b054438aa96d38b384908018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:31 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:42366
I20260806 03:54:31.890476 4005 tablet_service.cc:1514] Processing DeleteTablet for tablet 1df7d5d9b054438aa96d38b384908018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:31 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:39934
I20260806 03:54:31.890484 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 3711
I20260806 03:54:31.907779 4372 tablet_replica.cc:330] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: stopping tablet replica
I20260806 03:54:31.908033 4371 tablet_replica.cc:330] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: stopping tablet replica
I20260806 03:54:31.908190 4371 raft_consensus.cc:2229] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:31.908322 4372 raft_consensus.cc:2229] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:54:31.908473 4372 raft_consensus.cc:2258] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:31.908622 4371 raft_consensus.cc:2258] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:31.909132 4372 ts_tablet_manager.cc:1905] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:31.909132 4371 ts_tablet_manager.cc:1905] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:31.910949 4372 ts_tablet_manager.cc:1918] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:31.910949 4371 ts_tablet_manager.cc:1918] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:31.911041 4371 log.cc:1196] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/1df7d5d9b054438aa96d38b384908018
I20260806 03:54:31.911118 4372 log.cc:1196] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/1df7d5d9b054438aa96d38b384908018
I20260806 03:54:31.911387 4371 ts_tablet_manager.cc:1939] T 1df7d5d9b054438aa96d38b384908018 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Deleting consensus metadata
I20260806 03:54:31.911422 4372 ts_tablet_manager.cc:1939] T 1df7d5d9b054438aa96d38b384908018 P e4fc80fa33324e25b48b5a14e953001c: Deleting consensus metadata
I20260806 03:54:31.911816 3607 catalog_manager.cc:4841] TS e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289): tablet 1df7d5d9b054438aa96d38b384908018 (table test-table [id=4e97ec1893974006ae13edf5463581ac]) successfully deleted
I20260806 03:54:31.911898 3604 catalog_manager.cc:4841] TS 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505): tablet 1df7d5d9b054438aa96d38b384908018 (table test-table [id=4e97ec1893974006ae13edf5463581ac]) successfully deleted
W20260806 03:54:31.912631 3605 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:40919 (error 108)
I20260806 03:54:31.912942 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 3912
I20260806 03:54:31.923944 4333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:31.923801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40919 (local address 127.0.0.1:43838)
0806 03:54:31.923852 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:31.923900 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 03:54:31.924079 3605 catalog_manager.cc:4584] TS e50e26fef5d745de87690e08ad830ea9 (127.12.15.65:40919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1df7d5d9b054438aa96d38b384908018: Network error: Client connection negotiation failed: client connection to 127.12.15.65:40919: connect: Connection refused (error 111)
I20260806 03:54:31.928974 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 4111
I20260806 03:54:31.936904 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 3576
I20260806 03:54:51.944401 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33579
--webserver_interface=127.12.15.126
--webserver_port=43649
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:54:52.025390 4374 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:52.025588 4374 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:52.025616 4374 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:52.026777 4374 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 03:54:52.026820 4374 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 03:54:52.026839 4374 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 03:54:52.026854 4374 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:54:52.026870 4374 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:52.026894 4374 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:52.026917 4374 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:54:52.026942 4374 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:54:52.028422 4374 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:33579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4374
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:52.028669 4374 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:52.028872 4374 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:54:52.037808 4374 server_base.cc:1034] running on GCE node
W20260806 03:54:52.037839 4383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:52.037880 4385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:52.038050 4382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:52.038378 4374 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:52.038684 4374 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:52.039825 4374 hybrid_clock.cc:648] HybridClock initialized: now 1785988492039812 us; error 27 us; skew 500 ppm
Aug 06 03:54:52 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988492, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:52.042320 4374 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:54:52.042708 4374 webserver.cc:460] Webserver started at http://127.12.15.126:43649/ using document root <none> and password file <none>
I20260806 03:54:52.042897 4374 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:52.042948 4374 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:52.044184 4374 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:52.044775 4395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.044945 4374 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:52.045029 4374 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "dd1b66b4b2664aea9b3ba37186861db4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "6743d71a0c3a202a303eb977fde7a409"
server_key_iv: "83ab38f99913aa6dd7b554ab63030367"
server_key_version: "encryptionkey@0"
I20260806 03:54:52.045404 4374 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:52.052704 4374 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:52.054706 4374 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:52.054809 4374 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:52.058488 4374 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:33579
I20260806 03:54:52.058521 4474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:33579 every 8 connection(s)
I20260806 03:54:52.058864 4374 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:54:52.059913 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 4374
I20260806 03:54:52.060182 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:40919
--local_ip_for_outbound_sockets=127.12.15.65
--tserver_master_addrs=127.12.15.126:33579
--webserver_port=42861
--webserver_interface=127.12.15.65
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:52.060392 4475 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:54:52.061019 4475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Bootstrap starting.
I20260806 03:54:52.069222 4475 log.cc:824] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:52.071035 4475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:54:52.071290 4475 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Bootstrap complete.
I20260806 03:54:52.073331 4475 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:54:52.073550 4475 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1b66b4b2664aea9b3ba37186861db4, State: Initialized, Role: FOLLOWER
I20260806 03:54:52.073690 4475 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:54:52.073772 4475 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:54:52.073827 4475 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:54:52.073876 4475 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 1 FOLLOWER]: Advancing to term 2
I20260806 03:54:52.074916 4475 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:54:52.075017 4475 leader_election.cc:302] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd1b66b4b2664aea9b3ba37186861db4; no voters:
I20260806 03:54:52.075182 4475 leader_election.cc:288] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260806 03:54:52.075240 4482 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 2 FOLLOWER]: Leader election won for term 2
I20260806 03:54:52.075417 4482 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 2 LEADER]: Becoming Leader. State: Replica: dd1b66b4b2664aea9b3ba37186861db4, State: Running, Role: LEADER
I20260806 03:54:52.075460 4475 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:54:52.075579 4482 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:54:52.075940 4484 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd1b66b4b2664aea9b3ba37186861db4. Latest consensus state: current_term: 2 leader_uuid: "dd1b66b4b2664aea9b3ba37186861db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } } }
I20260806 03:54:52.075948 4483 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dd1b66b4b2664aea9b3ba37186861db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } } }
I20260806 03:54:52.076033 4484 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:52.076058 4483 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:54:52.076242 4491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:54:52.077090 4491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:54:52.077248 4491 catalog_manager.cc:1259] Loaded cluster ID: 0d6868d3b96f4aefbd68f826e14f623c
I20260806 03:54:52.077489 4491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:54:52.078655 4491 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:54:52.078835 4491 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Loaded TSK: 0
I20260806 03:54:52.096372 4491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Generated new TSK 1
I20260806 03:54:52.096470 4491 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:54:52.142309 4478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:52.142503 4478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:52.142534 4478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:52.143644 4478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:52.143684 4478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:52.143718 4478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:54:52.145151 4478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:40919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=42861
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:52.145381 4478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:52.145591 4478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:52.151520 4514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:52.151538 4519 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:52.151700 4521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:52.151863 4478 server_base.cc:1034] running on GCE node
I20260806 03:54:52.152072 4478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:52.152379 4478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:52.153568 4478 hybrid_clock.cc:648] HybridClock initialized: now 1785988492153550 us; error 37 us; skew 500 ppm
Aug 06 03:54:52 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988492, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:52.156730 4478 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:54:52.157300 4478 webserver.cc:460] Webserver started at http://127.12.15.65:42861/ using document root <none> and password file <none>
I20260806 03:54:52.157581 4478 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:52.157662 4478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:52.159533 4478 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:52.160336 4532 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.160526 4478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:52.160627 4478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "e50e26fef5d745de87690e08ad830ea9"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "a2b53233b9f2c00296657b61ca5041d8"
server_key_iv: "8f35c38ffaceea6052a7c179592a981f"
server_key_version: "encryptionkey@0"
I20260806 03:54:52.161070 4478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:52.196907 4478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:52.198994 4478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:52.199113 4478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:52.199373 4478 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:52.199883 4540 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 03:54:52.201315 4478 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:54:52.201367 4478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 03:54:52.201393 4478 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:54:52.202011 4478 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:54:52.202056 4478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 03:54:52.202144 4540 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Bootstrap starting.
I20260806 03:54:52.209636 4478 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:40919
I20260806 03:54:52.209719 4717 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:40919 every 8 connection(s)
I20260806 03:54:52.210155 4478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:54:52.213290 4540 log.cc:824] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:52.215690 4540 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:54:52.215963 4540 tablet_bootstrap.cc:492] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Bootstrap complete.
I20260806 03:54:52.216045 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 4478
I20260806 03:54:52.216351 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:44505
--local_ip_for_outbound_sockets=127.12.15.66
--tserver_master_addrs=127.12.15.126:33579
--webserver_port=39913
--webserver_interface=127.12.15.66
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:52.217013 4540 ts_tablet_manager.cc:1397] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.008s
Aug 06 03:54:52 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988492, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:52.218041 4540 raft_consensus.cc:348] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.218372 4540 raft_consensus.cc:729] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e50e26fef5d745de87690e08ad830ea9, State: Initialized, Role: FOLLOWER
I20260806 03:54:52.218509 4540 consensus_queue.cc:260] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.218803 4540 ts_tablet_manager.cc:1428] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260806 03:54:52.221875 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.211056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:39869 (local address 127.12.15.126:33579)
0806 03:54:52.211862 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.211869 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.211894 (+ 25us) server_negotiation.cc:407] Connection header received
0806 03:54:52.211992 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.211996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.212034 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.212131 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:52.213094 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.214160 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.215551 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.216012 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.216165 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.218657 (+ 2492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:52.218661 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:52.218661 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:52.218673 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:52.220206 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.220662 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.220664 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.220664 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.220752 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.221009 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.221010 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.221010 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.221222 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:52.221282 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.221632 (+ 350us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.221735 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":763,"thread_start_us":90,"threads_started":1}
I20260806 03:54:52.221937 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.210967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:39869)
0806 03:54:52.211189 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.211237 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:52.211692 (+ 455us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:52.212236 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:52.212241 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:52.212656 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:52.212961 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.212969 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.214275 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.214277 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.215470 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.215473 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.216166 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.216167 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.216321 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.216826 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:52.216835 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:52.218553 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:52.220283 (+ 1730us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:52.220291 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:52.220294 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:52.220588 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:52.220813 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:52.220814 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:52.220814 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:52.220950 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:52.221291 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:52.221299 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:52.221457 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 03:54:52.221577 (+ 120us) client_negotiation.cc:240] Negotiation successful
0806 03:54:52.221742 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":114,"threads_started":1}
I20260806 03:54:52.222632 4718 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33579
I20260806 03:54:52.222723 4718 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:52.222888 4718 heartbeater.cc:511] Master 127.12.15.126:33579 requested a full tablet report, sending...
I20260806 03:54:52.223639 4421 ts_manager.cc:194] Registered new tserver with Master: e50e26fef5d745de87690e08ad830ea9 (127.12.15.65:40919)
I20260806 03:54:52.224851 4421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:39869
I20260806 03:54:52.225863 4718 heartbeater.cc:503] Master 127.12.15.126:33579 was elected leader, sending a full tablet report...
Aug 06 03:54:52 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988492, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.65@KRBTEST.COM
I20260806 03:54:52.231406 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.224324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40919 (local address 127.0.0.1:34222)
0806 03:54:52.224551 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.224578 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:52.224631 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:52.224856 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:52.224862 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:52.225161 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:52.225222 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.225238 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.226077 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.226078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.226679 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.226681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.227120 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.227121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.227190 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.227614 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:52.227619 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:52.228839 (+ 1220us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:52.230376 (+ 1537us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:52.230382 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:52.230384 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:52.230573 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:52.230754 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:52.230755 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:52.230756 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:52.230813 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:52.231054 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:52.231061 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:52.231112 (+ 51us) client_negotiation.cc:769] Sending connection context
0806 03:54:52.231184 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:54:52.231308 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":98,"threads_started":1}
I20260806 03:54:52.231406 4732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.224410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34222 (local address 127.12.15.65:40919)
0806 03:54:52.224725 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.224728 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.224735 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:54:52.224763 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.224764 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.224776 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.224834 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:52.225308 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.225973 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.226762 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.227054 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.227112 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.228917 (+ 1805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:52.228925 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:52.228925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:52.228932 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:52.230304 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.230645 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.230648 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.230648 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.230700 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.230863 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.230864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.230864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.230992 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:52.231063 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.231227 (+ 164us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.231316 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290,"thread_start_us":63,"threads_started":1}
I20260806 03:54:52.231766 4608 tablet_service.cc:1514] Processing DeleteTablet for tablet 1df7d5d9b054438aa96d38b384908018 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:31 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:34222
I20260806 03:54:52.232244 4733 tablet_replica.cc:330] stopping tablet replica
I20260806 03:54:52.232357 4733 raft_consensus.cc:2229] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:52.232421 4733 raft_consensus.cc:2258] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:52.232714 4733 ts_tablet_manager.cc:1905] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:52.235266 4733 ts_tablet_manager.cc:1918] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:52.235344 4733 log.cc:1196] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal/wals/1df7d5d9b054438aa96d38b384908018
I20260806 03:54:52.235555 4733 ts_tablet_manager.cc:1939] T 1df7d5d9b054438aa96d38b384908018 P e50e26fef5d745de87690e08ad830ea9: Deleting consensus metadata
I20260806 03:54:52.236075 4397 catalog_manager.cc:4841] TS e50e26fef5d745de87690e08ad830ea9 (127.12.15.65:40919): tablet 1df7d5d9b054438aa96d38b384908018 (table test-table [id=4e97ec1893974006ae13edf5463581ac]) successfully deleted
W20260806 03:54:52.301455 4725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:52.301644 4725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:52.301666 4725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:52.302846 4725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:52.302889 4725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:52.302914 4725 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:54:52.304306 4725 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:44505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=39913
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:52.304512 4725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:52.304721 4725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:52.309651 4746 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:52.309640 4749 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:52.309643 4747 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:52.309850 4725 server_base.cc:1034] running on GCE node
I20260806 03:54:52.310063 4725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:52.310336 4725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:52.311482 4725 hybrid_clock.cc:648] HybridClock initialized: now 1785988492311460 us; error 37 us; skew 500 ppm
Aug 06 03:54:52 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988492, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:52.313910 4725 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:54:52.314365 4725 webserver.cc:460] Webserver started at http://127.12.15.66:39913/ using document root <none> and password file <none>
I20260806 03:54:52.314582 4725 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:52.314647 4725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:52.315903 4725 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:52.316424 4760 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.316576 4725 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:54:52.316650 4725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "7f926da79c511b9d495c52f481302704"
server_key_iv: "905704251723c8776d03a2859f98edcc"
server_key_version: "encryptionkey@0"
I20260806 03:54:52.316977 4725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:52.338757 4725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:52.340555 4725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:52.340689 4725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:52.340917 4725 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:52.341207 4725 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:52.341239 4725 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.341266 4725 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:52.341282 4725 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.346890 4725 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:44505
I20260806 03:54:52.346953 4919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:44505 every 8 connection(s)
I20260806 03:54:52.347275 4725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
Aug 06 03:54:52 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988492, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:52.353274 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 4725
I20260806 03:54:52.353602 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:46289
--local_ip_for_outbound_sockets=127.12.15.67
--tserver_master_addrs=127.12.15.126:33579
--webserver_port=45635
--webserver_interface=127.12.15.67
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:54:52.356168 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.347983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:56159 (local address 127.12.15.126:33579)
0806 03:54:52.348027 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.348029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.348245 (+ 216us) server_negotiation.cc:407] Connection header received
0806 03:54:52.348722 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.348723 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.348738 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.348780 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:52.349704 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.350253 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.351183 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.351402 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.351452 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.353515 (+ 2063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:52.353517 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:52.353517 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:52.353524 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:52.354779 (+ 1255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.355235 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.355236 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.355236 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.355285 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.355547 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.355547 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.355548 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.355682 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:52.355761 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.355988 (+ 227us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.356078 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:54:52.356312 4927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.347954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:56159)
0806 03:54:52.348117 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.348159 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:52.348628 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:52.348866 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:52.348869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:52.349431 (+ 562us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:52.349624 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.349629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.350322 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.350322 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.351112 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.351115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.351458 (+ 343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.351458 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.351556 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.351969 (+ 413us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:52.351977 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:52.353422 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:52.354854 (+ 1432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:52.354863 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:52.354866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:52.355169 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:52.355334 (+ 165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:52.355335 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:52.355335 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:52.355480 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:52.355754 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:52.355758 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:52.355875 (+ 117us) client_negotiation.cc:769] Sending connection context
0806 03:54:52.355955 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:54:52.356082 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":75,"threads_started":1}
I20260806 03:54:52.356767 4920 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33579
I20260806 03:54:52.356853 4920 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:52.357016 4920 heartbeater.cc:511] Master 127.12.15.126:33579 requested a full tablet report, sending...
I20260806 03:54:52.357422 4421 ts_manager.cc:194] Registered new tserver with Master: 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505)
I20260806 03:54:52.357880 4421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:56159
W20260806 03:54:52.436486 4928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:54:52.436697 4928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:54:52.436719 4928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:54:52.437812 4928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:54:52.437845 4928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:54:52.437870 4928 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:54:52.439287 4928 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:46289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=45635
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:54:52.439488 4928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:54:52.439726 4928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:54:52.444681 4936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:52.444705 4935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:54:52.444685 4940 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:54:52.444713 4928 server_base.cc:1034] running on GCE node
I20260806 03:54:52.445108 4928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:54:52.445353 4928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:54:52.446486 4928 hybrid_clock.cc:648] HybridClock initialized: now 1785988492446476 us; error 28 us; skew 500 ppm
Aug 06 03:54:52 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988492, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:54:52.448938 4928 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:54:52.449390 4928 webserver.cc:460] Webserver started at http://127.12.15.67:45635/ using document root <none> and password file <none>
I20260806 03:54:52.449589 4928 fs_manager.cc:359] Metadata directory not provided
I20260806 03:54:52.449651 4928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:54:52.450997 4928 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:54:52.451512 4948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.451670 4928 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.451747 4928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "e4fc80fa33324e25b48b5a14e953001c"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "79a60ca7922433b4c9c7477685787319"
server_key_iv: "894cfc7d0ed6a4db480e0d3900e22556"
server_key_version: "encryptionkey@0"
I20260806 03:54:52.452068 4928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:54:52.469599 4928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:54:52.471539 4928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:54:52.471674 4928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:54:52.471894 4928 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:54:52.472186 4928 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:54:52.472219 4928 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.472249 4928 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:54:52.472273 4928 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.477991 4928 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:46289
I20260806 03:54:52.478055 5121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:46289 every 8 connection(s)
I20260806 03:54:52.478395 4928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:54:52.480290 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 4928
Aug 06 03:54:52 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988492, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:54:52.486810 5127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.480727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59178 (local address 127.12.15.126:33579)
0806 03:54:52.480842 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.480844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.480899 (+ 55us) server_negotiation.cc:407] Connection header received
0806 03:54:52.480966 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.480967 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.480977 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.481029 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:52.481598 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.482079 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.482813 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.483008 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.483062 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.484137 (+ 1075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:52.484138 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:52.484139 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:52.484153 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:52.485777 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.486080 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.486081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.486081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.486168 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.486329 (+ 161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.486329 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.486330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.486439 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:52.486594 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.486637 (+ 43us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.486714 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95,"thread_start_us":66,"threads_started":1}
I20260806 03:54:52.487490 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.479085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:54323 (local address 127.12.15.126:33579)
0806 03:54:52.479134 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.479136 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.479352 (+ 216us) server_negotiation.cc:407] Connection header received
0806 03:54:52.479827 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.479828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.479838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.479897 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:52.480992 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.481494 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.482513 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.482711 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.482766 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.485025 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:52.485026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:52.485026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:52.485033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:52.486206 (+ 1173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.486549 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.486550 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.486550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.486604 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.486862 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.486863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.486864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.486991 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:52.487056 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.487333 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.487411 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:54:52.487759 5124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.479062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:54323)
0806 03:54:52.479232 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.479274 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:52.479749 (+ 475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:52.479950 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:52.479954 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:52.480651 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:54:52.480904 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.480910 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.481574 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.481575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.482439 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.482442 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.482779 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.482780 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.482905 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.483383 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:54:52.483393 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:54:52.484949 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:54:52.486269 (+ 1320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:52.486276 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:52.486278 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:52.486485 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:52.486663 (+ 178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:54:52.486664 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:54:52.486664 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:54:52.486768 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:54:52.487053 (+ 285us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:54:52.487060 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:54:52.487203 (+ 143us) client_negotiation.cc:769] Sending connection context
0806 03:54:52.487312 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 03:54:52.487461 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":80,"threads_started":1}
I20260806 03:54:52.488119 5122 heartbeater.cc:347] Connected to a master server at 127.12.15.126:33579
I20260806 03:54:52.488210 5122 heartbeater.cc:464] Registering TS with master...
I20260806 03:54:52.488379 5122 heartbeater.cc:511] Master 127.12.15.126:33579 requested a full tablet report, sending...
I20260806 03:54:52.488735 4421 ts_manager.cc:194] Registered new tserver with Master: e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
I20260806 03:54:52.489048 4421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:54323
I20260806 03:54:52.498451 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:54:52.500865 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.498796 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59190 (local address 127.12.15.126:33579)
0806 03:54:52.498833 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.498834 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.498927 (+ 93us) server_negotiation.cc:407] Connection header received
0806 03:54:52.498988 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.498988 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.499029 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.499093 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:54:52.499175 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.499619 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.500321 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.500506 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.500557 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.500673 (+ 116us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:54:52.500715 (+ 42us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0806 03:54:52.500817 (+ 102us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:59190: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 03:54:52.500906 4723 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:59190: token expired
W20260806 03:54:52.502204 4322 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.12.15.126:33579)
I20260806 03:54:52.507354 4723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.502435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59206 (local address 127.12.15.126:33579)
0806 03:54:52.502470 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.502472 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.502479 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:54:52.502497 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.502498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.502506 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.502559 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:54:52.502959 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.503384 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.503998 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.504177 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.504228 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.505167 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:54:52.505168 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:54:52.505168 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:54:52.505173 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:54:52.506503 (+ 1330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.506820 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.506821 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.506821 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.506870 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:54:52.507010 (+ 140us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:54:52.507011 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:54:52.507011 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:54:52.507105 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:54:52.507173 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.507241 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.507293 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:54:52.508466 4421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59206:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:54:52.509972 4421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:54:52.512048 4608 tablet_service.cc:1467] Processing CreateTablet for tablet 8416c4c032e04fd496df6ec1bfb0d9a1 (DEFAULT_TABLE table=test-table [id=c9067efb941f43ec8c92ac2178f38496]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:52.512262 4608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8416c4c032e04fd496df6ec1bfb0d9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:52.513609 4540 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Bootstrap starting.
I20260806 03:54:52.513887 4540 tablet_bootstrap.cc:654] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:52.515733 4540 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: No bootstrap required, opened a new log
I20260806 03:54:52.515784 5130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.511796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:46289 (local address 127.0.0.1:45516)
0806 03:54:52.511960 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.511970 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:52.512019 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:52.512479 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:52.512480 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:52.512482 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:52.512544 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.512546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.513754 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.513755 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.514622 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.514624 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.515501 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.515501 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.515573 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.515600 (+ 27us) client_negotiation.cc:769] Sending connection context
0806 03:54:52.515662 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:54:52.515713 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":118,"threads_started":1}
I20260806 03:54:52.515790 4540 ts_tablet_manager.cc:1397] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260806 03:54:52.515910 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.512051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.12.15.67:46289)
0806 03:54:52.512245 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.512249 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.512261 (+ 12us) server_negotiation.cc:407] Connection header received
0806 03:54:52.512299 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.512301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.512317 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.512392 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:52.512628 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.513662 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.514701 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.515429 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.515503 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.515567 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.515679 (+ 112us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.515723 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":92,"threads_started":1}
I20260806 03:54:52.515952 4540 raft_consensus.cc:348] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.516031 4540 raft_consensus.cc:374] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:52.516086 4540 raft_consensus.cc:729] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e50e26fef5d745de87690e08ad830ea9, State: Initialized, Role: FOLLOWER
I20260806 03:54:52.516134 4540 consensus_queue.cc:260] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.516227 4540 ts_tablet_manager.cc:1428] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260806 03:54:52.516758 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.511706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:44505 (local address 127.0.0.1:47992)
0806 03:54:52.511753 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.511762 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:52.511797 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:52.512313 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:52.512314 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:52.512315 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:52.512359 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.512360 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.514393 (+ 2033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.514394 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.515483 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.515486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.516552 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.516553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.516626 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.516630 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:54:52.516657 (+ 27us) client_negotiation.cc:240] Negotiation successful
0806 03:54:52.516699 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 03:54:52.516942 5019 tablet_service.cc:1467] Processing CreateTablet for tablet 8416c4c032e04fd496df6ec1bfb0d9a1 (DEFAULT_TABLE table=test-table [id=c9067efb941f43ec8c92ac2178f38496]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:52.517079 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.511805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47992 (local address 127.12.15.66:44505)
0806 03:54:52.512132 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.512146 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.512160 (+ 14us) server_negotiation.cc:407] Connection header received
0806 03:54:52.512209 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.512212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.512227 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.512307 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:52.512421 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.513506 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.515571 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.516440 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.516867 (+ 427us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.516934 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.516962 (+ 28us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.517001 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":107,"threads_started":1}
I20260806 03:54:52.517272 5019 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8416c4c032e04fd496df6ec1bfb0d9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:52.517719 4820 tablet_service.cc:1467] Processing CreateTablet for tablet 8416c4c032e04fd496df6ec1bfb0d9a1 (DEFAULT_TABLE table=test-table [id=c9067efb941f43ec8c92ac2178f38496]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:54:52.518045 4820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8416c4c032e04fd496df6ec1bfb0d9a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:54:52.520780 5136 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Bootstrap starting.
I20260806 03:54:52.521425 5136 tablet_bootstrap.cc:654] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:52.521449 5138 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Bootstrap starting.
I20260806 03:54:52.522011 5136 log.cc:824] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:52.522048 5138 tablet_bootstrap.cc:654] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Neither blocks nor log segments found. Creating new log.
I20260806 03:54:52.522610 5138 log.cc:824] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:54:52.522814 5136 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: No bootstrap required, opened a new log
I20260806 03:54:52.522874 5136 ts_tablet_manager.cc:1397] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:52.523253 5138 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: No bootstrap required, opened a new log
I20260806 03:54:52.523306 5138 ts_tablet_manager.cc:1397] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:54:52.524189 5136 raft_consensus.cc:348] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.524319 5136 raft_consensus.cc:374] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:52.524351 5136 raft_consensus.cc:729] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e4fc80fa33324e25b48b5a14e953001c, State: Initialized, Role: FOLLOWER
I20260806 03:54:52.524457 5136 consensus_queue.cc:260] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.524689 5136 ts_tablet_manager.cc:1428] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:52.524729 5122 heartbeater.cc:503] Master 127.12.15.126:33579 was elected leader, sending a full tablet report...
I20260806 03:54:52.524657 5138 raft_consensus.cc:348] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.524832 5138 raft_consensus.cc:374] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:54:52.524875 5138 raft_consensus.cc:729] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e98a89ba4bf4a0da198f39bbe34a3c4, State: Initialized, Role: FOLLOWER
I20260806 03:54:52.524982 5138 consensus_queue.cc:260] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.525262 5138 ts_tablet_manager.cc:1428] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:54:52.525267 4920 heartbeater.cc:503] Master 127.12.15.126:33579 was elected leader, sending a full tablet report...
W20260806 03:54:52.597795 4923 tablet.cc:2369] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:54:52.693223 4729 raft_consensus.cc:482] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:54:52.693363 4729 raft_consensus.cc:504] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.693727 4729 leader_election.cc:288] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
I20260806 03:54:52.697736 4722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.693898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:46289 (local address 127.12.15.65:40085)
0806 03:54:52.694033 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.694046 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:52.694138 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:52.694295 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:52.694295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:52.694297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:52.694398 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.694400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.695357 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.695357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.696647 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.696650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.697429 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.697430 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.697546 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.697557 (+ 11us) client_negotiation.cc:769] Sending connection context
0806 03:54:52.697605 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 03:54:52.697647 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":24}
I20260806 03:54:52.697891 5133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.693965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:40085 (local address 127.12.15.67:46289)
0806 03:54:52.694015 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.694017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.694091 (+ 74us) server_negotiation.cc:407] Connection header received
0806 03:54:52.694211 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.694212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.694222 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.694282 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:52.694505 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.695263 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.696728 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.697352 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.697429 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.697439 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.697778 (+ 339us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.697822 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:54:52.697992 5145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.693900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:44505 (local address 127.12.15.65:58181)
0806 03:54:52.694061 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.694069 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:54:52.694141 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:54:52.694295 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:54:52.694295 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:54:52.694297 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:54:52.694398 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.694400 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.695357 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.695357 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.696647 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:54:52.696650 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.697444 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.697445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:54:52.697519 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.697554 (+ 35us) client_negotiation.cc:769] Sending connection context
0806 03:54:52.697636 (+ 82us) client_negotiation.cc:240] Negotiation successful
0806 03:54:52.697667 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"mutex_wait_us":120,"thread_start_us":71,"threads_started":1}
I20260806 03:54:52.697832 5129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.693968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:58181 (local address 127.12.15.66:44505)
0806 03:54:52.694042 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.694044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.694148 (+ 104us) server_negotiation.cc:407] Connection header received
0806 03:54:52.694211 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.694212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.694223 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.694282 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:54:52.694504 (+ 222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.695289 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.696769 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.697318 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.697364 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.697375 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.697655 (+ 280us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.697730 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:54:52.698356 4854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8416c4c032e04fd496df6ec1bfb0d9a1" candidate_uuid: "e50e26fef5d745de87690e08ad830ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" is_pre_election: true
I20260806 03:54:52.698522 4854 raft_consensus.cc:2454] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e50e26fef5d745de87690e08ad830ea9 in term 0.
I20260806 03:54:52.698506 5055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8416c4c032e04fd496df6ec1bfb0d9a1" candidate_uuid: "e50e26fef5d745de87690e08ad830ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4fc80fa33324e25b48b5a14e953001c" is_pre_election: true
I20260806 03:54:52.698717 5055 raft_consensus.cc:2454] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e50e26fef5d745de87690e08ad830ea9 in term 0.
I20260806 03:54:52.698765 4533 leader_election.cc:302] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4e98a89ba4bf4a0da198f39bbe34a3c4, e50e26fef5d745de87690e08ad830ea9; no voters:
I20260806 03:54:52.698873 4729 raft_consensus.cc:2781] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:54:52.698925 4729 raft_consensus.cc:482] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:54:52.698948 4729 raft_consensus.cc:3037] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:52.699676 4729 raft_consensus.cc:504] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.699800 4729 leader_election.cc:288] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 1 election: Requested vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
I20260806 03:54:52.699944 4854 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8416c4c032e04fd496df6ec1bfb0d9a1" candidate_uuid: "e50e26fef5d745de87690e08ad830ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4"
I20260806 03:54:52.699982 5055 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8416c4c032e04fd496df6ec1bfb0d9a1" candidate_uuid: "e50e26fef5d745de87690e08ad830ea9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e4fc80fa33324e25b48b5a14e953001c"
I20260806 03:54:52.700065 5055 raft_consensus.cc:3037] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:52.700088 4854 raft_consensus.cc:3037] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:54:52.700922 5055 raft_consensus.cc:2454] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e50e26fef5d745de87690e08ad830ea9 in term 1.
I20260806 03:54:52.701105 4854 raft_consensus.cc:2454] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e50e26fef5d745de87690e08ad830ea9 in term 1.
I20260806 03:54:52.701151 4536 leader_election.cc:302] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e4fc80fa33324e25b48b5a14e953001c, e50e26fef5d745de87690e08ad830ea9; no voters:
I20260806 03:54:52.701256 4729 raft_consensus.cc:2781] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:54:52.701450 4729 raft_consensus.cc:686] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 LEADER]: Becoming Leader. State: Replica: e50e26fef5d745de87690e08ad830ea9, State: Running, Role: LEADER
I20260806 03:54:52.701534 4729 consensus_queue.cc:237] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:54:52.702926 4421 catalog_manager.cc:5495] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 reported cstate change: term changed from 0 to 1, leader changed from <none> to e50e26fef5d745de87690e08ad830ea9 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "e50e26fef5d745de87690e08ad830ea9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } health_report { overall_health: HEALTHY } } }
W20260806 03:54:52.710943 4721 tablet.cc:2369] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:54:52.728891 5123 tablet.cc:2369] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:54:52.741528 5156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.739440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34228 (local address 127.12.15.65:40919)
0806 03:54:52.739548 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0806 03:54:52.739549 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:54:52.739562 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:54:52.739596 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:54:52.739597 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:54:52.739616 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:54:52.739684 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:54:52.739789 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.740261 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.740920 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:54:52.741105 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:54:52.741147 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:54:52.741254 (+ 107us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:54:52.741353 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:54:52.741401 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:54:52.741430 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 03:54:52.741447 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":59,"threads_started":1}
I20260806 03:54:52.742992 5055 raft_consensus.cc:1261] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 1 FOLLOWER]: Refusing update from remote peer e50e26fef5d745de87690e08ad830ea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:52.742992 4854 raft_consensus.cc:1261] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 1 FOLLOWER]: Refusing update from remote peer e50e26fef5d745de87690e08ad830ea9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:54:52.743314 4729 consensus_queue.cc:1035] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:52.743449 5149 consensus_queue.cc:1035] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:54:52.746322 5161 mvcc.cc:205] Tried to move back new op lower bound from 7315408866272878592 to 7315408866106126336. Current Snapshot: MvccSnapshot[applied={T|T < 7315408866272878592 or (T in {7315408866272878592})}]
I20260806 03:54:52.747044 5162 mvcc.cc:205] Tried to move back new op lower bound from 7315408866272878592 to 7315408866106126336. Current Snapshot: MvccSnapshot[applied={T|T < 7315408866272878592 or (T in {7315408866272878592})}]
I20260806 03:54:52.747802 4420 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:52.747902 4420 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59206:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:54:52.748582 4420 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Sending DeleteTablet for 3 replicas of tablet 8416c4c032e04fd496df6ec1bfb0d9a1
I20260806 03:54:52.748886 4820 tablet_service.cc:1514] Processing DeleteTablet for tablet 8416c4c032e04fd496df6ec1bfb0d9a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:52 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:47992
I20260806 03:54:52.748896 5019 tablet_service.cc:1514] Processing DeleteTablet for tablet 8416c4c032e04fd496df6ec1bfb0d9a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:52 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:45516
I20260806 03:54:52.749109 5173 tablet_replica.cc:330] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: stopping tablet replica
I20260806 03:54:52.749096 4608 tablet_service.cc:1514] Processing DeleteTablet for tablet 8416c4c032e04fd496df6ec1bfb0d9a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:54:52 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:34222
I20260806 03:54:52.749208 5173 raft_consensus.cc:2229] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:52.749313 5173 raft_consensus.cc:2258] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:52.749326 5176 tablet_replica.cc:330] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: stopping tablet replica
I20260806 03:54:52.749420 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 4478
I20260806 03:54:52.749457 5176 raft_consensus.cc:2229] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:54:52.749752 5176 raft_consensus.cc:2258] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:52.754804 5173 ts_tablet_manager.cc:1905] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:52.756313 5173 ts_tablet_manager.cc:1918] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:52.756381 5173 log.cc:1196] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/8416c4c032e04fd496df6ec1bfb0d9a1
I20260806 03:54:52.756611 5173 ts_tablet_manager.cc:1939] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e4fc80fa33324e25b48b5a14e953001c: Deleting consensus metadata
I20260806 03:54:52.757167 4398 catalog_manager.cc:4841] TS e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289): tablet 8416c4c032e04fd496df6ec1bfb0d9a1 (table test-table [id=c9067efb941f43ec8c92ac2178f38496]) successfully deleted
I20260806 03:54:52.757491 5175 tablet_replica.cc:330] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: stopping tablet replica
I20260806 03:54:52.757596 5175 raft_consensus.cc:2229] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:54:52.757688 5175 raft_consensus.cc:2258] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:54:52.758172 5175 ts_tablet_manager.cc:1905] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:54:52.760011 5175 ts_tablet_manager.cc:1918] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:54:52.760135 5175 log.cc:1196] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/8416c4c032e04fd496df6ec1bfb0d9a1
I20260806 03:54:52.760408 5175 ts_tablet_manager.cc:1939] T 8416c4c032e04fd496df6ec1bfb0d9a1 P 4e98a89ba4bf4a0da198f39bbe34a3c4: Deleting consensus metadata
I20260806 03:54:52.760792 4396 catalog_manager.cc:4841] TS 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505): tablet 8416c4c032e04fd496df6ec1bfb0d9a1 (table test-table [id=c9067efb941f43ec8c92ac2178f38496]) successfully deleted
W20260806 03:54:52.764212 4397 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:40919 (error 108)
I20260806 03:54:52.764478 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 4725
I20260806 03:54:52.766898 4730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:54:52.764561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:40919 (local address 127.0.0.1:34230)
0806 03:54:52.766800 (+ 2239us) negotiation.cc:107] Waiting for socket to connect
0806 03:54:52.766862 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:40919: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2211}
W20260806 03:54:52.767019 4397 catalog_manager.cc:4584] TS e50e26fef5d745de87690e08ad830ea9 (127.12.15.65:40919): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8416c4c032e04fd496df6ec1bfb0d9a1: Network error: Client connection negotiation failed: client connection to 127.12.15.65:40919: connect: Connection refused (error 111)
I20260806 03:54:52.775008 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 4928
I20260806 03:54:52.784448 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 4374
I20260806 03:55:12.794984 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33579
--webserver_interface=127.12.15.126
--webserver_port=43649
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:33579
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:12.876153 5178 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:12.876351 5178 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:12.876381 5178 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:12.877502 5178 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260806 03:55:12.877542 5178 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260806 03:55:12.877558 5178 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260806 03:55:12.877573 5178 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:12.877588 5178 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:12.877604 5178 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:12.877627 5178 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:12.877650 5178 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:12.879088 5178 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:33579
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33579
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=43649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5178
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:12.879330 5178 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:12.879526 5178 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:55:12.884375 5178 server_base.cc:1034] running on GCE node
W20260806 03:55:12.884341 5187 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:12.884341 5191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:12.884361 5185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:12.884819 5178 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:12.885068 5178 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:12.886193 5178 hybrid_clock.cc:648] HybridClock initialized: now 1785988512886175 us; error 27 us; skew 500 ppm
Aug 06 03:55:12 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988512, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:12.888578 5178 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:55:12.888999 5178 webserver.cc:460] Webserver started at http://127.12.15.126:43649/ using document root <none> and password file <none>
I20260806 03:55:12.889178 5178 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:12.889216 5178 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:12.890424 5178 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:55:12.891016 5200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:12.891167 5178 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:12.891247 5178 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "dd1b66b4b2664aea9b3ba37186861db4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "6743d71a0c3a202a303eb977fde7a409"
server_key_iv: "83ab38f99913aa6dd7b554ab63030367"
server_key_version: "encryptionkey@0"
I20260806 03:55:12.891554 5178 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:12.900974 5178 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:12.902621 5178 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:12.902719 5178 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:12.906322 5178 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:33579
I20260806 03:55:12.906399 5277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:33579 every 8 connection(s)
I20260806 03:55:12.906706 5178 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:12.908159 5280 sys_catalog.cc:263] Verifying existing consensus state
I20260806 03:55:12.908829 5280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Bootstrap starting.
I20260806 03:55:12.910444 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 5178
I20260806 03:55:12.910797 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:40919
--local_ip_for_outbound_sockets=127.12.15.65
--tserver_master_addrs=127.12.15.126:33579
--webserver_port=42861
--webserver_interface=127.12.15.65
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:55:12.917888 5280 log.cc:824] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:12.920197 5280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:55:12.920490 5280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Bootstrap complete.
I20260806 03:55:12.922730 5280 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:55:12.922961 5280 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd1b66b4b2664aea9b3ba37186861db4, State: Initialized, Role: FOLLOWER
I20260806 03:55:12.923133 5280 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:55:12.923218 5280 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:12.923265 5280 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:12.923316 5280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 2 FOLLOWER]: Advancing to term 3
I20260806 03:55:12.924311 5280 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:55:12.924409 5280 leader_election.cc:302] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd1b66b4b2664aea9b3ba37186861db4; no voters:
I20260806 03:55:12.924576 5280 leader_election.cc:288] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260806 03:55:12.924626 5284 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 3 FOLLOWER]: Leader election won for term 3
I20260806 03:55:12.924768 5284 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [term 3 LEADER]: Becoming Leader. State: Replica: dd1b66b4b2664aea9b3ba37186861db4, State: Running, Role: LEADER
I20260806 03:55:12.924849 5280 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:12.924926 5284 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } }
I20260806 03:55:12.925217 5284 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd1b66b4b2664aea9b3ba37186861db4. Latest consensus state: current_term: 3 leader_uuid: "dd1b66b4b2664aea9b3ba37186861db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } } }
I20260806 03:55:12.925586 5284 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:12.925504 5286 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "dd1b66b4b2664aea9b3ba37186861db4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd1b66b4b2664aea9b3ba37186861db4" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33579 } } }
I20260806 03:55:12.925977 5286 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:12.926139 5303 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:12.926952 5303 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:12.927037 5303 catalog_manager.cc:1259] Loaded cluster ID: 0d6868d3b96f4aefbd68f826e14f623c
I20260806 03:55:12.927057 5303 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:12.928421 5303 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:12.928596 5303 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Loaded TSK: 0
I20260806 03:55:12.928629 5303 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dd1b66b4b2664aea9b3ba37186861db4: Loaded TSK: 1
F20260806 03:55:12.928848 5303 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fbe06c729ad google::LogMessage::Fail() at ??:0
@ 0x7fbe06c7588a google::LogMessage::SendToLog() at ??:0
@ 0x7fbe06c724de google::LogMessage::Flush() at ??:0
@ 0x7fbe06c73a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fbe093df104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fbe093dcd41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fbe0ebcc1cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fbe0ebcc48e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fbe0ebcc594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fbe0eb92c48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fbe0eb9cad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fbe08b400d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fbe08b35954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fbe099996db start_thread at ??:0
@ 0x7fbe0577061f clone at ??:0
W20260806 03:55:13.012516 5282 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:13.012712 5282 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:13.012734 5282 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:13.013844 5282 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:13.013881 5282 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:13.013906 5282 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:55:13.015338 5282 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:40919
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=42861
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5282
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:13.015548 5282 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:13.015765 5282 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:13.021517 5351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:13.021587 5350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:13.021852 5282 server_base.cc:1034] running on GCE node
W20260806 03:55:13.021523 5353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:13.022190 5282 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:13.022490 5282 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:13.023679 5282 hybrid_clock.cc:648] HybridClock initialized: now 1785988513023660 us; error 31 us; skew 500 ppm
Aug 06 03:55:13 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988513, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:13.026826 5282 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:55:13.027366 5282 webserver.cc:460] Webserver started at http://127.12.15.65:42861/ using document root <none> and password file <none>
I20260806 03:55:13.027642 5282 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:13.027719 5282 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:13.029458 5282 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260806 03:55:13.030207 5362 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.030383 5282 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.030467 5282 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "e50e26fef5d745de87690e08ad830ea9"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "a2b53233b9f2c00296657b61ca5041d8"
server_key_iv: "8f35c38ffaceea6052a7c179592a981f"
server_key_version: "encryptionkey@0"
I20260806 03:55:13.030790 5282 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:13.053208 5282 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:13.055398 5282 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:13.055537 5282 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:13.055757 5282 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:13.056192 5372 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260806 03:55:13.058203 5282 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260806 03:55:13.058248 5282 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260806 03:55:13.058283 5282 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260806 03:55:13.058897 5282 ts_tablet_manager.cc:610] Registered 1 tablets
I20260806 03:55:13.058930 5282 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260806 03:55:13.059036 5372 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Bootstrap starting.
I20260806 03:55:13.065177 5282 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:40919
I20260806 03:55:13.065325 5540 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:40919 every 8 connection(s)
I20260806 03:55:13.065603 5282 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:55:13.067286 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 5282
I20260806 03:55:13.067543 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:44505
--local_ip_for_outbound_sockets=127.12.15.66
--tserver_master_addrs=127.12.15.126:33579
--webserver_port=39913
--webserver_interface=127.12.15.66
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:55:13.069973 5372 log.cc:824] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:13.072115 5372 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260806 03:55:13.072386 5372 tablet_bootstrap.cc:492] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Bootstrap complete.
I20260806 03:55:13.073388 5372 ts_tablet_manager.cc:1397] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Time spent bootstrapping tablet: real 0.014s user 0.005s sys 0.008s
I20260806 03:55:13.074851 5372 raft_consensus.cc:348] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:55:13.075182 5372 raft_consensus.cc:729] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e50e26fef5d745de87690e08ad830ea9, State: Initialized, Role: FOLLOWER
I20260806 03:55:13.075353 5372 consensus_queue.cc:260] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:55:13.075656 5372 ts_tablet_manager.cc:1428] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
W20260806 03:55:13.120893 5543 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:55:13.066273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:50383)
0806 03:55:13.066439 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.066483 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:13.066929 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:13.120714 (+ 53785us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":76,"threads_started":1}
W20260806 03:55:13.121194 5541 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:33579 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.126:33579: Client connection negotiation failed: client connection to 127.12.15.126:33579: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260806 03:55:13.121553 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:13.121438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:53439)
0806 03:55:13.121485 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.121531 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 03:55:13.121878 5543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:13.121796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:37995)
0806 03:55:13.121834 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.121857 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 03:55:13.121987 5541 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260806 03:55:13.155928 5544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:13.156140 5544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:13.156172 5544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:13.157275 5544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:13.157315 5544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:13.157348 5544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:55:13.158818 5544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:44505
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=39913
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5544
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:13.159049 5544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:13.159253 5544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:13.164052 5552 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:13.164048 5559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:13.164050 5555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:13.164145 5544 server_base.cc:1034] running on GCE node
I20260806 03:55:13.164410 5544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:13.164641 5544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:13.165776 5544 hybrid_clock.cc:648] HybridClock initialized: now 1785988513165763 us; error 28 us; skew 500 ppm
Aug 06 03:55:13 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988513, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:13.168166 5544 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:55:13.168588 5544 webserver.cc:460] Webserver started at http://127.12.15.66:39913/ using document root <none> and password file <none>
I20260806 03:55:13.168779 5544 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:13.168828 5544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:13.170063 5544 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:55:13.170631 5566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.170783 5544 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.170866 5544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "7f926da79c511b9d495c52f481302704"
server_key_iv: "905704251723c8776d03a2859f98edcc"
server_key_version: "encryptionkey@0"
I20260806 03:55:13.171187 5544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:13.194854 5544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:13.196645 5544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:13.196780 5544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:13.196995 5544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:13.197301 5544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:13.197337 5544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.197367 5544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:13.197388 5544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.203203 5544 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:44505
I20260806 03:55:13.203264 5734 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:44505 every 8 connection(s)
I20260806 03:55:13.203580 5544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:55:13.204588 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:13.204215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:53301)
0806 03:55:13.204363 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.204436 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
W20260806 03:55:13.204978 5735 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:33579 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.126:33579: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
I20260806 03:55:13.205243 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:13.205151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:41291)
0806 03:55:13.205188 (+ 37us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.205215 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 03:55:13.205637 5737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:13.205521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:38795)
0806 03:55:13.205582 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.205614 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260806 03:55:13.205755 5735 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:55:13.206590 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 5544
I20260806 03:55:13.206876 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:46289
--local_ip_for_outbound_sockets=127.12.15.67
--tserver_master_addrs=127.12.15.126:33579
--webserver_port=45635
--webserver_interface=127.12.15.67
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38381
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:13.289594 5738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:13.289793 5738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:13.289816 5738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:13.290972 5738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:13.291013 5738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:13.291039 5738 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:55:13.292447 5738 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38381
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:46289
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=45635
--tserver_master_addrs=127.12.15.126:33579
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:13.292677 5738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:13.292886 5738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:13.297694 5752 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:13.297708 5750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:13.297753 5748 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:13.297737 5738 server_base.cc:1034] running on GCE node
I20260806 03:55:13.297998 5738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:13.298250 5738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:13.299398 5738 hybrid_clock.cc:648] HybridClock initialized: now 1785988513299385 us; error 29 us; skew 500 ppm
Aug 06 03:55:13 dist-test-slave-bqcl krb5kdc[3552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988513, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:13.301759 5738 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:55:13.302203 5738 webserver.cc:460] Webserver started at http://127.12.15.67:45635/ using document root <none> and password file <none>
I20260806 03:55:13.302378 5738 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:13.302418 5738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:13.303647 5738 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:55:13.304159 5768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.304315 5738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:13.304399 5738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "e4fc80fa33324e25b48b5a14e953001c"
format_stamp: "Formatted at 2026-08-06 03:54:31 on dist-test-slave-bqcl"
server_key: "79a60ca7922433b4c9c7477685787319"
server_key_iv: "894cfc7d0ed6a4db480e0d3900e22556"
server_key_version: "encryptionkey@0"
I20260806 03:55:13.304719 5738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:13.320698 5738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:13.322779 5738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:13.322887 5738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:13.323072 5738 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:13.323318 5738 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:13.323348 5738 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.323367 5738 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:13.323383 5738 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:13.328866 5738 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:46289
I20260806 03:55:13.328908 5935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:46289 every 8 connection(s)
I20260806 03:55:13.329218 5738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:55:13.330284 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:13.329862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:36061)
0806 03:55:13.329995 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.330060 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100,"thread_start_us":67,"threads_started":1}
W20260806 03:55:13.330652 5936 heartbeater.cc:644] Failed to heartbeat to 127.12.15.126:33579 (0 consecutive failures): Network error: Failed to ping master at 127.12.15.126:33579: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
I20260806 03:55:13.330875 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:13.330798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:58091)
0806 03:55:13.330834 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.330859 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 03:55:13.331076 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:13.331016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:59029)
0806 03:55:13.331047 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:13.331065 (+ 18us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
W20260806 03:55:13.331159 5936 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260806 03:55:13.332317 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 5738
I20260806 03:55:14.122567 5941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:14.122330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:56605)
0806 03:55:14.122457 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:14.122496 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":64,"threads_started":1}
I20260806 03:55:14.206357 5942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:14.206116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:59107)
0806 03:55:14.206261 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:14.206300 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
I20260806 03:55:14.331700 5944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:14.331472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:45509)
0806 03:55:14.331612 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:14.331649 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":88,"threads_started":1}
I20260806 03:55:14.731237 5945 raft_consensus.cc:482] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:14.731364 5945 raft_consensus.cc:504] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:55:14.731670 5945 leader_election.cc:288] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
Aug 06 03:55:14 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988513, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.66@KRBTEST.COM
Aug 06 03:55:14 dist-test-slave-bqcl krb5kdc[3552](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988513, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.67@KRBTEST.COM
I20260806 03:55:14.742283 5946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:14.731822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:44505 (local address 127.12.15.65:41485)
0806 03:55:14.732000 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:14.732011 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:14.732102 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:14.732899 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:14.732902 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:14.733368 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:14.733547 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:14.733553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:14.734688 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:14.734689 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:14.735483 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:14.735485 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:14.736198 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:14.736199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:14.736257 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:14.736826 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:14.736830 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:14.738553 (+ 1723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:14.740517 (+ 1964us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:14.740524 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:14.740525 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:14.740808 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:14.741084 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:14.741085 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:14.741085 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:14.741220 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:14.741719 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:14.741723 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:14.741850 (+ 127us) client_negotiation.cc:769] Sending connection context
0806 03:55:14.741963 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 03:55:14.742146 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":45,"thread_start_us":77,"threads_started":1}
I20260806 03:55:14.742297 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:14.731857 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:46289 (local address 127.12.15.65:58987)
0806 03:55:14.732030 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:14.732041 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:14.732110 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:14.732900 (+ 790us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:14.732903 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:14.733366 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:14.733547 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:14.733553 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:14.734715 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:14.734716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:14.735481 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:14.735483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:14.736129 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:14.736130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:14.736256 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:14.736794 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:14.736805 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:14.738927 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:14.740711 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:14.740712 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:14.740713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:14.740907 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:14.741137 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:14.741137 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:14.741137 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:14.741220 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:14.741804 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:14.741805 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:14.741850 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 03:55:14.741963 (+ 113us) client_negotiation.cc:240] Negotiation successful
0806 03:55:14.742198 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"mutex_wait_us":169,"thread_start_us":93,"threads_started":1}
I20260806 03:55:14.742332 5952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:14.732027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:58987 (local address 127.12.15.67:46289)
0806 03:55:14.732196 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:14.732208 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:14.732236 (+ 28us) server_negotiation.cc:407] Connection header received
0806 03:55:14.732672 (+ 436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:14.732674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:14.732716 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:14.732848 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:14.733669 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:14.734636 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:14.735563 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:14.736001 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:14.736120 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:14.739018 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:14.739021 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:14.739021 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:14.739031 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:14.740642 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:14.740972 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:14.740975 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:14.740975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:14.741067 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:14.741310 (+ 243us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:14.741312 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:14.741312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:14.741743 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:14.741860 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:14.742040 (+ 180us) server_negotiation.cc:299] Negotiation successful
0806 03:55:14.742214 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":56,"threads_started":1}
I20260806 03:55:14.742889 5949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:14.731999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:41485 (local address 127.12.15.66:44505)
0806 03:55:14.732197 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:14.732209 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:14.732229 (+ 20us) server_negotiation.cc:407] Connection header received
0806 03:55:14.732687 (+ 458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:14.732689 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:14.732727 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:14.732847 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:14.733665 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:14.734607 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:14.735567 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:14.736023 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:14.736117 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:14.738662 (+ 2545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:14.738665 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:14.738666 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:14.738676 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:14.740441 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:14.740879 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:14.740897 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:14.740897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:14.741007 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:14.741311 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:14.741312 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:14.741312 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:14.741647 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:14.741743 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:14.742051 (+ 308us) server_negotiation.cc:299] Negotiation successful
0806 03:55:14.742209 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":54,"threads_started":1}
W20260806 03:55:14.743044 5365 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
W20260806 03:55:14.743520 5363 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
I20260806 03:55:14.743582 5363 leader_election.cc:302] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e50e26fef5d745de87690e08ad830ea9; no voters: 4e98a89ba4bf4a0da198f39bbe34a3c4, e4fc80fa33324e25b48b5a14e953001c
I20260806 03:55:14.743708 5945 raft_consensus.cc:2726] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:55:15.123159 5947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:15.123026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:48043)
0806 03:55:15.123084 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:15.123114 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260806 03:55:15.206988 5954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:15.206746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:40925)
0806 03:55:15.206871 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:15.206921 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":59,"threads_started":1}
I20260806 03:55:15.332314 5957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:15.332110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:52877)
0806 03:55:15.332238 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:15.332272 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260806 03:55:16.123804 5958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:16.123578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:41393)
0806 03:55:16.123715 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:16.123753 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":67,"threads_started":1}
I20260806 03:55:16.207654 5959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:16.207450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:50573)
0806 03:55:16.207577 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:16.207612 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":76,"threads_started":1}
I20260806 03:55:16.332901 5962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:16.332703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:45379)
0806 03:55:16.332829 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:16.332860 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":62,"threads_started":1}
I20260806 03:55:16.598078 5963 raft_consensus.cc:482] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:16.598214 5963 raft_consensus.cc:504] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:55:16.598402 5963 leader_election.cc:288] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
W20260806 03:55:16.598851 5363 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
W20260806 03:55:16.598964 5365 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
I20260806 03:55:16.599025 5365 leader_election.cc:302] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e50e26fef5d745de87690e08ad830ea9; no voters: 4e98a89ba4bf4a0da198f39bbe34a3c4, e4fc80fa33324e25b48b5a14e953001c
I20260806 03:55:16.599108 5963 raft_consensus.cc:2726] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:55:17.124465 5964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:17.124240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:60727)
0806 03:55:17.124377 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:17.124413 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260806 03:55:17.208310 5966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:17.208050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:58789)
0806 03:55:17.208189 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:17.208246 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":76,"threads_started":1}
I20260806 03:55:17.333470 5967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:17.333266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:47865)
0806 03:55:17.333391 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:17.333428 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":58,"threads_started":1}
I20260806 03:55:18.125165 5970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:18.124926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:48949)
0806 03:55:18.125074 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:18.125116 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":79,"threads_started":1}
I20260806 03:55:18.208940 5972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:18.208708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:35049)
0806 03:55:18.208856 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:18.208897 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":74,"threads_started":1}
I20260806 03:55:18.334057 5974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:18.333863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:43109)
0806 03:55:18.333974 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:18.334013 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96,"thread_start_us":62,"threads_started":1}
I20260806 03:55:19.052606 5975 raft_consensus.cc:482] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:19.052721 5975 raft_consensus.cc:504] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:55:19.052866 5975 leader_election.cc:288] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
W20260806 03:55:19.053264 5365 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
W20260806 03:55:19.053359 5363 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
I20260806 03:55:19.053398 5363 leader_election.cc:302] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e50e26fef5d745de87690e08ad830ea9; no voters: 4e98a89ba4bf4a0da198f39bbe34a3c4, e4fc80fa33324e25b48b5a14e953001c
I20260806 03:55:19.053460 5975 raft_consensus.cc:2726] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:55:19.125833 5976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:19.125597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:41551)
0806 03:55:19.125748 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:19.125781 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260806 03:55:19.209578 5977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:19.209369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:37827)
0806 03:55:19.209499 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:19.209532 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":77,"threads_started":1}
I20260806 03:55:19.334671 5978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:19.334456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:48877)
0806 03:55:19.334585 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:19.334623 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":73,"threads_started":1}
I20260806 03:55:20.126473 5979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:20.126245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:51959)
0806 03:55:20.126388 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:20.126423 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":87,"threads_started":1}
I20260806 03:55:20.210206 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:20.209968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:52691)
0806 03:55:20.210106 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:20.210155 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":63,"threads_started":1}
I20260806 03:55:20.335261 5983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:20.335060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:39259)
0806 03:55:20.335177 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:20.335214 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":65,"threads_started":1}
I20260806 03:55:21.127089 5986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:21.126877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:57829)
0806 03:55:21.127000 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:21.127038 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":69,"threads_started":1}
I20260806 03:55:21.210871 5987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:21.210653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:40379)
0806 03:55:21.210790 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:21.210827 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":69,"threads_started":1}
I20260806 03:55:21.335856 5988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:21.335635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:51571)
0806 03:55:21.335776 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:21.335813 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":65,"threads_started":1}
I20260806 03:55:21.875634 5989 raft_consensus.cc:482] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:21.875747 5989 raft_consensus.cc:504] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:55:21.875921 5989 leader_election.cc:288] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
W20260806 03:55:21.876273 5363 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
W20260806 03:55:21.876368 5365 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
I20260806 03:55:21.876415 5365 leader_election.cc:302] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e50e26fef5d745de87690e08ad830ea9; no voters: 4e98a89ba4bf4a0da198f39bbe34a3c4, e4fc80fa33324e25b48b5a14e953001c
I20260806 03:55:21.876475 5989 raft_consensus.cc:2726] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:55:22.127730 5990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:22.127492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:49861)
0806 03:55:22.127630 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:22.127668 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":67,"threads_started":1}
I20260806 03:55:22.211467 5992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:22.211269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:57167)
0806 03:55:22.211388 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:22.211423 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":63,"threads_started":1}
I20260806 03:55:22.336488 5994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:22.336253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:52223)
0806 03:55:22.336402 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:22.336445 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":77,"threads_started":1}
I20260806 03:55:23.128432 5995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:23.128192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:57909)
0806 03:55:23.128344 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:23.128380 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
I20260806 03:55:23.212085 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:23.211850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:48625)
0806 03:55:23.212002 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:23.212041 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260806 03:55:23.337050 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:23.336849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:47231)
0806 03:55:23.336971 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:23.337005 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260806 03:55:24.129074 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:24.128847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:50269)
0806 03:55:24.128980 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:24.129025 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":69,"threads_started":1}
I20260806 03:55:24.212729 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:24.212504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:33691)
0806 03:55:24.212643 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:24.212683 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":64,"threads_started":1}
I20260806 03:55:24.337683 6000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:24.337453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:44791)
0806 03:55:24.337598 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:24.337636 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":81,"threads_started":1}
I20260806 03:55:25.129737 6001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:25.129496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:46095)
0806 03:55:25.129648 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:25.129685 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
I20260806 03:55:25.213383 6002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:25.213149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:33947)
0806 03:55:25.213301 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:25.213338 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260806 03:55:25.338305 6003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:25.338071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:51041)
0806 03:55:25.338221 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:25.338259 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":92,"threads_started":1}
I20260806 03:55:26.130456 6007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:26.130275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:55079)
0806 03:55:26.130380 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:26.130416 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":57,"threads_started":1}
I20260806 03:55:26.190163 6008 raft_consensus.cc:482] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:26.190276 6008 raft_consensus.cc:504] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4e98a89ba4bf4a0da198f39bbe34a3c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 44505 } } peers { permanent_uuid: "e4fc80fa33324e25b48b5a14e953001c" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 46289 } } peers { permanent_uuid: "e50e26fef5d745de87690e08ad830ea9" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 40919 } }
I20260806 03:55:26.190430 6008 leader_election.cc:288] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505), e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289)
W20260806 03:55:26.190824 5363 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4e98a89ba4bf4a0da198f39bbe34a3c4 (127.12.15.66:44505): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
W20260806 03:55:26.190917 5365 leader_election.cc:341] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e4fc80fa33324e25b48b5a14e953001c (127.12.15.67:46289): Not found: Tablet not found: 8416c4c032e04fd496df6ec1bfb0d9a1
I20260806 03:55:26.190989 5365 leader_election.cc:302] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e50e26fef5d745de87690e08ad830ea9; no voters: 4e98a89ba4bf4a0da198f39bbe34a3c4, e4fc80fa33324e25b48b5a14e953001c
I20260806 03:55:26.191064 6008 raft_consensus.cc:2726] T 8416c4c032e04fd496df6ec1bfb0d9a1 P e50e26fef5d745de87690e08ad830ea9 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260806 03:55:26.214044 6009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:26.213810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:57939)
0806 03:55:26.213953 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:26.213993 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":62,"threads_started":1}
I20260806 03:55:26.338891 6012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:26.338691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:48159)
0806 03:55:26.338808 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:26.338845 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":62,"threads_started":1}
I20260806 03:55:27.131054 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:27.130852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:45825)
0806 03:55:27.130974 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:27.131010 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":64,"threads_started":1}
I20260806 03:55:27.214694 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:27.214483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:52313)
0806 03:55:27.214610 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:27.214646 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":75,"threads_started":1}
I20260806 03:55:27.339594 6015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:27.339310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:48403)
0806 03:55:27.339465 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:27.339533 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":64,"threads_started":1}
I20260806 03:55:28.131683 6016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:28.131491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.65:58351)
0806 03:55:28.131609 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:28.131643 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":66,"threads_started":1}
I20260806 03:55:28.215302 6017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:28.215119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.66:40241)
0806 03:55:28.215227 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:28.215257 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":64,"threads_started":1}
I20260806 03:55:28.334307 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 5282
I20260806 03:55:28.341152 6018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:28.340802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:33579 (local address 127.12.15.67:38215)
0806 03:55:28.341058 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:28.341100 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.126:33579: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":125,"threads_started":1}
I20260806 03:55:28.345417 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 5544
I20260806 03:55:28.355619 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 5738
2026-08-06T03:55:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58508 ms)
[----------] 30 tests from SecurityITest (233029 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:29 dist-test-slave-bqcl krb5kdc[6022](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:29 dist-test-slave-bqcl krb5kdc[6022](info): set up 2 sockets
Aug 06 03:55:29 dist-test-slave-bqcl krb5kdc[6022](info): commencing operation
krb5kdc: starting...
W20260806 03:55:30.294641 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.918s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:55:30.413702 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37169
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:37169
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:30.496445 6043 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:30.496658 6043 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:30.496680 6043 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:30.497809 6043 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:30.497848 6043 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:30.497861 6043 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:30.497885 6043 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:30.497906 6043 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:30.499385 6043 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:37169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6043
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:30.499619 6043 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:30.499832 6043 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:30.504532 6050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:30.504557 6054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:30.504734 6052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:30.504865 6043 server_base.cc:1034] running on GCE node
I20260806 03:55:30.505103 6043 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:30.505370 6043 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:30.506502 6043 hybrid_clock.cc:648] HybridClock initialized: now 1785988530506489 us; error 30 us; skew 500 ppm
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:30.508944 6043 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:55:30.509322 6043 webserver.cc:460] Webserver started at http://127.12.15.126:38627/ using document root <none> and password file <none>
I20260806 03:55:30.509505 6043 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:30.509559 6043 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:30.509686 6043 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:30.510703 6043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "86b1bd063c66460786d9a9bce08d5143"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "b3f14579355e66d75d26a48aab5706ff"
server_key_iv: "07232b95e36b70f5bcde900e402cb4a4"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.511027 6043 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "86b1bd063c66460786d9a9bce08d5143"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "b3f14579355e66d75d26a48aab5706ff"
server_key_iv: "07232b95e36b70f5bcde900e402cb4a4"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.512306 6043 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:55:30.512966 6065 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.513119 6043 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:30.513199 6043 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "86b1bd063c66460786d9a9bce08d5143"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "b3f14579355e66d75d26a48aab5706ff"
server_key_iv: "07232b95e36b70f5bcde900e402cb4a4"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.513257 6043 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:30.524444 6043 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:30.526265 6043 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:30.526397 6043 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:30.529994 6043 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:37169
I20260806 03:55:30.530035 6141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:37169 every 8 connection(s)
I20260806 03:55:30.530356 6043 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:30.531132 6142 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:30.533272 6142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143: Bootstrap starting.
I20260806 03:55:30.533802 6142 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:30.534032 6142 log.cc:824] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:30.534686 6142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143: No bootstrap required, opened a new log
I20260806 03:55:30.536172 6142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b1bd063c66460786d9a9bce08d5143" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37169 } }
I20260806 03:55:30.536311 6142 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:30.536334 6142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86b1bd063c66460786d9a9bce08d5143, State: Initialized, Role: FOLLOWER
I20260806 03:55:30.536434 6142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b1bd063c66460786d9a9bce08d5143" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37169 } }
I20260806 03:55:30.536482 6142 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:30.536507 6142 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:30.536540 6142 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:30.536983 6142 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b1bd063c66460786d9a9bce08d5143" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37169 } }
I20260806 03:55:30.537071 6142 leader_election.cc:302] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86b1bd063c66460786d9a9bce08d5143; no voters:
I20260806 03:55:30.537214 6142 leader_election.cc:288] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:30.537263 6145 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:30.537421 6142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:30.537439 6145 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [term 1 LEADER]: Becoming Leader. State: Replica: 86b1bd063c66460786d9a9bce08d5143, State: Running, Role: LEADER
I20260806 03:55:30.537621 6145 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b1bd063c66460786d9a9bce08d5143" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37169 } }
I20260806 03:55:30.537984 6147 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86b1bd063c66460786d9a9bce08d5143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b1bd063c66460786d9a9bce08d5143" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37169 } } }
I20260806 03:55:30.538064 6147 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:30.538106 6157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86b1bd063c66460786d9a9bce08d5143. Latest consensus state: current_term: 1 leader_uuid: "86b1bd063c66460786d9a9bce08d5143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86b1bd063c66460786d9a9bce08d5143" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37169 } } }
I20260806 03:55:30.538178 6157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:30.538187 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 6043
I20260806 03:55:30.538287 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:30.538498 12349 external_mini_cluster.cc:1428] Setting key 99db6f531f744cfd770c8ea0817d2cd5
W20260806 03:55:30.539376 6178 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:55:30.539422 6178 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:55:30.539511 6170 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:30.539682 6170 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:30.541026 6170 catalog_manager.cc:1347] Generated new cluster ID: 3a629d021b814bebb15809b0cd8e6642
I20260806 03:55:30.541076 6170 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:30.546420 6170 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:30.547011 6170 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:30.549860 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:30.539192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39446 (local address 127.12.15.126:37169)
0806 03:55:30.539331 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:30.539341 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:30.539492 (+ 151us) server_negotiation.cc:407] Connection header received
0806 03:55:30.539637 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:30.539645 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:30.539703 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:30.539845 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:30.540566 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.541844 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.542965 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.543543 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.543636 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:30.546363 (+ 2727us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:30.546367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:30.546367 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:30.546379 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:30.548324 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:30.548701 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:30.548704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:30.548704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:30.548816 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:30.548987 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:30.548988 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:30.548988 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:30.549270 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:30.549418 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:30.549486 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 03:55:30.549646 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":70,"threads_started":1}
I20260806 03:55:30.553447 6170 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143: Generated new TSK 0
I20260806 03:55:30.553634 6170 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:55:30.580265 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37169
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:30.661233 6191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:30.661432 6191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:30.661455 6191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:30.662609 6191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:30.662652 6191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:30.662678 6191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:55:30.664104 6191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6191
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:30.664335 6191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:30.664558 6191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:30.669301 6204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:30.669410 6201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:30.669304 6200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:30.669586 6191 server_base.cc:1034] running on GCE node
I20260806 03:55:30.669813 6191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:30.670058 6191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:30.671229 6191 hybrid_clock.cc:648] HybridClock initialized: now 1785988530671211 us; error 34 us; skew 500 ppm
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:30.673774 6191 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:55:30.674273 6191 webserver.cc:460] Webserver started at http://127.12.15.65:38789/ using document root <none> and password file <none>
I20260806 03:55:30.674511 6191 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:30.674566 6191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:30.674706 6191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:30.675796 6191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "07bf17d898bb457889367a72562255de"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "8ac7b9de3982c6681abbdbb420b4b92b"
server_key_iv: "3248469f785bc92b391d04d9f6969c88"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.676153 6191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "07bf17d898bb457889367a72562255de"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "8ac7b9de3982c6681abbdbb420b4b92b"
server_key_iv: "3248469f785bc92b391d04d9f6969c88"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.677465 6191 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:30.678248 6215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.678408 6191 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:30.678520 6191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "07bf17d898bb457889367a72562255de"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "8ac7b9de3982c6681abbdbb420b4b92b"
server_key_iv: "3248469f785bc92b391d04d9f6969c88"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.678593 6191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:30.685972 6191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:30.687765 6191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:30.687919 6191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:30.688141 6191 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:30.688437 6191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:30.688483 6191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.688517 6191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:30.688545 6191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.694705 6191 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:39319
I20260806 03:55:30.694761 6382 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:39319 every 8 connection(s)
I20260806 03:55:30.695037 6191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:55:30.695262 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 6191
I20260806 03:55:30.695370 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:55:30.695618 12349 external_mini_cluster.cc:1428] Setting key a0ed93f413a8ec423091f19e0a9e9301
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:30.705827 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:30.695831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:57781 (local address 127.12.15.126:37169)
0806 03:55:30.695876 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:30.695879 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:30.696154 (+ 275us) server_negotiation.cc:407] Connection header received
0806 03:55:30.696772 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:30.696773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:30.696783 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:30.696836 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:30.698069 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.698756 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.700267 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.700638 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.700708 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:30.703204 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:30.703206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:30.703206 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:30.703213 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:30.704465 (+ 1252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:30.704910 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:30.704911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:30.704911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:30.704957 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:30.705219 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:30.705219 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:30.705220 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:30.705332 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:30.705380 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:30.705701 (+ 321us) server_negotiation.cc:299] Negotiation successful
0806 03:55:30.705764 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:55:30.706081 6386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:30.695829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37169 (local address 127.12.15.65:57781)
0806 03:55:30.696025 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:30.696074 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:30.696673 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:30.696909 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:30.696914 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:30.697672 (+ 758us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:30.697951 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:30.697958 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.698866 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.698868 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:30.700150 (+ 1282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:30.700153 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.700748 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.700749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:30.700912 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:30.701395 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:30.701405 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:30.703117 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:30.704542 (+ 1425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:30.704549 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:30.704551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:30.704840 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:30.705014 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:30.705014 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:30.705014 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:30.705153 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:30.705389 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:30.705395 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:30.705570 (+ 175us) client_negotiation.cc:769] Sending connection context
0806 03:55:30.705693 (+ 123us) client_negotiation.cc:240] Negotiation successful
0806 03:55:30.705862 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":96,"threads_started":1}
I20260806 03:55:30.706610 6384 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37169
I20260806 03:55:30.706700 6384 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:30.706868 6384 heartbeater.cc:511] Master 127.12.15.126:37169 requested a full tablet report, sending...
I20260806 03:55:30.707378 6094 ts_manager.cc:194] Registered new tserver with Master: 07bf17d898bb457889367a72562255de (127.12.15.65:39319)
I20260806 03:55:30.707952 6094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:57781
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:55:30.724906 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37169
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:30.806059 6391 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:30.806267 6391 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:30.806289 6391 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:30.807356 6391 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:30.807389 6391 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:30.807412 6391 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:55:30.808796 6391 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6391
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:30.808997 6391 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:30.809181 6391 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:30.813928 6402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:30.813910 6400 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:30.813953 6391 server_base.cc:1034] running on GCE node
W20260806 03:55:30.813894 6404 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:30.814337 6391 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:30.814581 6391 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:30.815702 6391 hybrid_clock.cc:648] HybridClock initialized: now 1785988530815688 us; error 30 us; skew 500 ppm
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:30.818070 6391 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:55:30.818471 6391 webserver.cc:460] Webserver started at http://127.12.15.66:45617/ using document root <none> and password file <none>
I20260806 03:55:30.818662 6391 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:30.818704 6391 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:30.818819 6391 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:30.819748 6391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "2b4d845966664b719aba495f4c97ab8c"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "7c4e92c0321c64a87e54e7e5880f7578"
server_key_iv: "5887008faf2a6451c7c61d785f4b99a8"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.820052 6391 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "2b4d845966664b719aba495f4c97ab8c"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "7c4e92c0321c64a87e54e7e5880f7578"
server_key_iv: "5887008faf2a6451c7c61d785f4b99a8"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.821244 6391 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:30.821897 6416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.822059 6391 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:30.822161 6391 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "2b4d845966664b719aba495f4c97ab8c"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "7c4e92c0321c64a87e54e7e5880f7578"
server_key_iv: "5887008faf2a6451c7c61d785f4b99a8"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.822220 6391 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:30.837105 6391 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:30.839171 6391 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:30.839322 6391 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:30.839517 6391 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:30.839798 6391 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:30.839831 6391 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.839862 6391 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:30.839884 6391 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.845176 6391 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:39857
I20260806 03:55:30.845250 6589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:39857 every 8 connection(s)
I20260806 03:55:30.845520 6391 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:55:30.849519 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 6391
I20260806 03:55:30.849622 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:55:30.849848 12349 external_mini_cluster.cc:1428] Setting key 5664b8ea18364e82547ecdcfa2255f52
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:30.854480 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:30.846198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:33267 (local address 127.12.15.126:37169)
0806 03:55:30.846255 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:30.846258 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:30.846429 (+ 171us) server_negotiation.cc:407] Connection header received
0806 03:55:30.846922 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:30.846923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:30.846936 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:30.846987 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:30.847901 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.848403 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.849363 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.849558 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.849622 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:30.851963 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:30.851964 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:30.851965 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:30.851971 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:30.853206 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:30.853569 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:30.853570 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:30.853571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:30.853629 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:30.853844 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:30.853845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:30.853845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:30.853956 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:30.854017 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:30.854308 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 03:55:30.854390 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:55:30.854650 6592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:30.846157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37169 (local address 127.12.15.66:33267)
0806 03:55:30.846318 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:30.846356 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:30.846826 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:30.847053 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:30.847057 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:30.847625 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:30.847806 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:30.847810 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.848486 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.848487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:30.849279 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:30.849282 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.849659 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.849661 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:30.849819 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:30.850313 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:30.850321 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:30.851876 (+ 1555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:30.853280 (+ 1404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:30.853287 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:30.853289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:30.853503 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:30.853682 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:30.853683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:30.853684 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:30.853788 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:30.854015 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:30.854022 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:30.854180 (+ 158us) client_negotiation.cc:769] Sending connection context
0806 03:55:30.854279 (+ 99us) client_negotiation.cc:240] Negotiation successful
0806 03:55:30.854432 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":81,"threads_started":1}
I20260806 03:55:30.855065 6590 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37169
I20260806 03:55:30.855168 6590 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:30.855351 6590 heartbeater.cc:511] Master 127.12.15.126:37169 requested a full tablet report, sending...
I20260806 03:55:30.855726 6094 ts_manager.cc:194] Registered new tserver with Master: 2b4d845966664b719aba495f4c97ab8c (127.12.15.66:39857)
I20260806 03:55:30.856213 6094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:33267
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:55:30.874979 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37169
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34435
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:30.955971 6601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:30.956172 6601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:30.956194 6601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:30.957315 6601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:30.957358 6601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:30.957386 6601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:55:30.958827 6601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34435
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:37169
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:30.959040 6601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:30.959252 6601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:55:30.964166 6601 server_base.cc:1034] running on GCE node
W20260806 03:55:30.964201 6610 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:30.964139 6609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:30.964131 6614 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:30.964564 6601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:30.964818 6601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:30.965950 6601 hybrid_clock.cc:648] HybridClock initialized: now 1785988530965940 us; error 28 us; skew 500 ppm
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:30.968411 6601 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:55:30.968842 6601 webserver.cc:460] Webserver started at http://127.12.15.67:40731/ using document root <none> and password file <none>
I20260806 03:55:30.969039 6601 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:30.969079 6601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:30.969180 6601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:30.970116 6601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "f44e603a6ab140ae8d452b7e12da1808"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "b50bbfbde045b68394a7c661fbe9b3d6"
server_key_iv: "133d347827fb966985493bd699fc84c9"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.970465 6601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "f44e603a6ab140ae8d452b7e12da1808"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "b50bbfbde045b68394a7c661fbe9b3d6"
server_key_iv: "133d347827fb966985493bd699fc84c9"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.971654 6601 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:55:30.972376 6624 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.972564 6601 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:55:30.972659 6601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "f44e603a6ab140ae8d452b7e12da1808"
format_stamp: "Formatted at 2026-08-06 03:55:30 on dist-test-slave-bqcl"
server_key: "b50bbfbde045b68394a7c661fbe9b3d6"
server_key_iv: "133d347827fb966985493bd699fc84c9"
server_key_version: "encryptionkey@0"
I20260806 03:55:30.972721 6601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:30.985370 6601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:30.987530 6601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:30.987643 6601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:30.987831 6601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:30.988102 6601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:30.988133 6601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.988155 6601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:30.988168 6601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:30.993526 6601 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:38977
I20260806 03:55:30.993619 6792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:38977 every 8 connection(s)
I20260806 03:55:30.993875 6601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
Aug 06 03:55:30 dist-test-slave-bqcl krb5kdc[6022](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988530, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:30.999547 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 6601
I20260806 03:55:30.999666 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:55:30.999857 12349 external_mini_cluster.cc:1428] Setting key 9f219597ca6f9ca9be8dec4bd1c399fc
I20260806 03:55:31.002490 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:30.994550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:37943 (local address 127.12.15.126:37169)
0806 03:55:30.994597 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:30.994599 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:30.994800 (+ 201us) server_negotiation.cc:407] Connection header received
0806 03:55:30.995253 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:30.995254 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:30.995263 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:30.995319 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:30.996219 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.996713 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.997639 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.997818 (+ 179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.997864 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:30.999905 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:30.999907 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:30.999907 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:30.999915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:31.001202 (+ 1287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:31.001563 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:31.001564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:31.001565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:31.001625 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:31.001837 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:31.001838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:31.001839 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:31.001952 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:31.002008 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.002324 (+ 316us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.002403 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:55:31.002645 6798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:30.994518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:37169 (local address 127.12.15.67:37943)
0806 03:55:30.994676 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:30.994717 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:30.995176 (+ 459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:30.995385 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:30.995389 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:30.995955 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:30.996141 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:30.996146 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.996775 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.996776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:30.997573 (+ 797us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:30.997575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:30.997871 (+ 296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:30.997872 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:30.997961 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:30.998430 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:30.998438 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:30.999818 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:31.001277 (+ 1459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:31.001284 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:31.001286 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:31.001497 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:31.001677 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:31.001678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:31.001678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:31.001783 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:31.002014 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:31.002021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:31.002189 (+ 168us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.002280 (+ 91us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.002431 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":73,"threads_started":1}
I20260806 03:55:31.003067 6795 heartbeater.cc:347] Connected to a master server at 127.12.15.126:37169
I20260806 03:55:31.003154 6795 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:31.003326 6795 heartbeater.cc:511] Master 127.12.15.126:37169 requested a full tablet report, sending...
I20260806 03:55:31.003674 6094 ts_manager.cc:194] Registered new tserver with Master: f44e603a6ab140ae8d452b7e12da1808 (127.12.15.67:38977)
I20260806 03:55:31.004009 6094 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:37943
I20260806 03:55:31.011559 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:55:31.018600 6183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.012877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39452 (local address 127.12.15.126:37169)
0806 03:55:31.012941 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.012943 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.013055 (+ 112us) server_negotiation.cc:407] Connection header received
0806 03:55:31.013114 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.013115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.013125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.013165 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:31.013731 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.014244 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.014962 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.015151 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.015204 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.016307 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:31.016308 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:31.016309 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:31.016316 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:31.017535 (+ 1219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:31.017836 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:31.017837 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:31.017837 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:31.017885 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:31.018045 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:31.018045 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:31.018046 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:31.018222 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:31.018307 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.018439 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.018514 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260806 03:55:31.019938 6094 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:55:31.021457 6094 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:55:31.027464 6811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.023402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:39319 (local address 127.0.0.1:38178)
0806 03:55:31.023576 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.023617 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.023680 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.024141 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.024144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.024146 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:31.024207 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.024209 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.025216 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.025217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.026311 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.026314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.027192 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.027193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.027270 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.027285 (+ 15us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.027342 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.027394 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":63,"threads_started":1}
I20260806 03:55:31.027506 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.023795 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.12.15.65:39319)
0806 03:55:31.023978 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.023983 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.023991 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:55:31.024033 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.024034 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.024047 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.024113 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:31.024296 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.025128 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.026404 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.027123 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.027290 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.027360 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.027386 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.027424 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":88,"threads_started":1}
I20260806 03:55:31.027889 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.023582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56944 (local address 127.12.15.67:38977)
0806 03:55:31.023996 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.024001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.024009 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:55:31.024037 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.024039 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.024050 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.024122 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:31.024381 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.025542 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.026719 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.027328 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.027393 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.027443 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.027776 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.027818 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":92,"threads_started":1}
I20260806 03:55:31.027968 6809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.023403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:39857 (local address 127.0.0.1:51426)
0806 03:55:31.023651 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.023661 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.023714 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.024469 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.024470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.024473 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:31.024520 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.024521 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.025845 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.025846 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.026908 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.026911 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.027750 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.027751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.027813 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.027817 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.027864 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.027901 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"spinlock_wait_cycles":384,"thread_start_us":104,"threads_started":1}
I20260806 03:55:31.027709 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.023402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38977 (local address 127.0.0.1:56944)
0806 03:55:31.023787 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.023795 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.023828 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.024241 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.024242 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.024245 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:31.024290 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.024292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.025686 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.025687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.026648 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.026651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.027478 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.027480 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.027546 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.027551 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.027599 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.027637 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":376,"mutex_wait_us":5,"thread_start_us":65,"threads_started":1}
I20260806 03:55:31.028189 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.023573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.12.15.66:39857)
0806 03:55:31.024259 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.024264 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.024273 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:55:31.024307 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.024309 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.024322 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.024392 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:31.024637 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.025740 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.026989 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.027674 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.027737 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.027791 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.028068 (+ 277us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.028116 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":72,"threads_started":1}
I20260806 03:55:31.028448 6283 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7 (DEFAULT_TABLE table=test-table [id=0fa201667b6c46f2ad1d8e4e7d85cc4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:31.028755 6497 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7 (DEFAULT_TABLE table=test-table [id=0fa201667b6c46f2ad1d8e4e7d85cc4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:31.028848 6283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:31.028843 6700 tablet_service.cc:1467] Processing CreateTablet for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7 (DEFAULT_TABLE table=test-table [id=0fa201667b6c46f2ad1d8e4e7d85cc4a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:31.029071 6497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:31.029161 6700 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:31.031497 6819 tablet_bootstrap.cc:492] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Bootstrap starting.
I20260806 03:55:31.032104 6822 tablet_bootstrap.cc:492] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Bootstrap starting.
I20260806 03:55:31.032142 6819 tablet_bootstrap.cc:654] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:31.032750 6819 log.cc:824] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:31.032801 6822 tablet_bootstrap.cc:654] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:31.033303 6822 log.cc:824] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:31.033527 6819 tablet_bootstrap.cc:492] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: No bootstrap required, opened a new log
I20260806 03:55:31.033600 6819 ts_tablet_manager.cc:1397] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:55:31.033656 6821 tablet_bootstrap.cc:492] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de: Bootstrap starting.
I20260806 03:55:31.033977 6822 tablet_bootstrap.cc:492] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: No bootstrap required, opened a new log
I20260806 03:55:31.034047 6822 ts_tablet_manager.cc:1397] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:31.034288 6821 tablet_bootstrap.cc:654] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:31.034821 6821 log.cc:824] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:31.035516 6819 raft_consensus.cc:348] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.035657 6819 raft_consensus.cc:374] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:31.035677 6819 raft_consensus.cc:729] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b4d845966664b719aba495f4c97ab8c, State: Initialized, Role: FOLLOWER
I20260806 03:55:31.035681 6822 raft_consensus.cc:348] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.035776 6819 consensus_queue.cc:260] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.035847 6822 raft_consensus.cc:374] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:31.035892 6822 raft_consensus.cc:729] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f44e603a6ab140ae8d452b7e12da1808, State: Initialized, Role: FOLLOWER
I20260806 03:55:31.036041 6819 ts_tablet_manager.cc:1428] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260806 03:55:31.036019 6822 consensus_queue.cc:260] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.036068 6590 heartbeater.cc:503] Master 127.12.15.126:37169 was elected leader, sending a full tablet report...
I20260806 03:55:31.036279 6822 ts_tablet_manager.cc:1428] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:55:31.036279 6795 heartbeater.cc:503] Master 127.12.15.126:37169 was elected leader, sending a full tablet report...
I20260806 03:55:31.036604 6821 tablet_bootstrap.cc:492] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de: No bootstrap required, opened a new log
I20260806 03:55:31.036677 6821 ts_tablet_manager.cc:1397] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260806 03:55:31.038419 6821 raft_consensus.cc:348] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.038604 6821 raft_consensus.cc:374] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:31.038686 6821 raft_consensus.cc:729] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07bf17d898bb457889367a72562255de, State: Initialized, Role: FOLLOWER
I20260806 03:55:31.038806 6821 consensus_queue.cc:260] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.039053 6821 ts_tablet_manager.cc:1428] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:55:31.039088 6384 heartbeater.cc:503] Master 127.12.15.126:37169 was elected leader, sending a full tablet report...
W20260806 03:55:31.096052 6591 tablet.cc:2369] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:55:31.195823 6385 tablet.cc:2369] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:55:31.244459 6796 tablet.cc:2369] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:55:31.376657 6826 raft_consensus.cc:482] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:31.376793 6826 raft_consensus.cc:504] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.377115 6826 leader_election.cc:288] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f44e603a6ab140ae8d452b7e12da1808 (127.12.15.67:38977), 07bf17d898bb457889367a72562255de (127.12.15.65:39319)
I20260806 03:55:31.380805 6830 raft_consensus.cc:482] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:31.380972 6817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.377385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:59985 (local address 127.12.15.65:39319)
0806 03:55:31.377434 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.377436 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.377606 (+ 170us) server_negotiation.cc:407] Connection header received
0806 03:55:31.377665 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.377666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.377679 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.377728 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:31.377908 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.378627 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.380076 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.380629 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.380705 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.380716 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.380874 (+ 158us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.380915 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:55:31.381012 6833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.377313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38977 (local address 127.12.15.66:41401)
0806 03:55:31.377466 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.377477 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.377541 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.377677 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.377678 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.377681 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:31.377756 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.377758 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.378668 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.378669 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.379977 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.379980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.380743 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.380744 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.380800 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.380802 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.380852 (+ 50us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.380901 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"mutex_wait_us":33,"thread_start_us":69,"threads_started":1}
I20260806 03:55:31.381012 6835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.377313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:39319 (local address 127.12.15.66:59985)
0806 03:55:31.377517 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.377526 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.377606 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.377735 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.377736 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.377738 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:31.377812 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.377831 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.378703 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.378704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.379977 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.379980 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.380707 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.380708 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.380794 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.380799 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.380853 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.380901 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":23,"thread_start_us":69,"threads_started":1}
I20260806 03:55:31.381137 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.377419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:41401 (local address 127.12.15.67:38977)
0806 03:55:31.377471 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.377474 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.377530 (+ 56us) server_negotiation.cc:407] Connection header received
0806 03:55:31.377612 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.377613 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.377622 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.377668 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:31.377815 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.378592 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.380070 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.380672 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.380727 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.380734 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.381025 (+ 291us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.381068 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:55:31.381472 6830 raft_consensus.cc:504] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.381572 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb89d0e5bea4e4fbf847e1c95f1f0b7" candidate_uuid: "2b4d845966664b719aba495f4c97ab8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07bf17d898bb457889367a72562255de" is_pre_election: true
I20260806 03:55:31.381572 6727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb89d0e5bea4e4fbf847e1c95f1f0b7" candidate_uuid: "2b4d845966664b719aba495f4c97ab8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44e603a6ab140ae8d452b7e12da1808" is_pre_election: true
I20260806 03:55:31.381680 6316 raft_consensus.cc:2454] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b4d845966664b719aba495f4c97ab8c in term 0.
I20260806 03:55:31.381711 6727 raft_consensus.cc:2454] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b4d845966664b719aba495f4c97ab8c in term 0.
I20260806 03:55:31.381935 6421 leader_election.cc:302] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b4d845966664b719aba495f4c97ab8c, f44e603a6ab140ae8d452b7e12da1808; no voters:
I20260806 03:55:31.382074 6826 raft_consensus.cc:2781] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:55:31.382102 6830 leader_election.cc:288] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f44e603a6ab140ae8d452b7e12da1808 (127.12.15.67:38977), 2b4d845966664b719aba495f4c97ab8c (127.12.15.66:39857)
I20260806 03:55:31.382143 6826 raft_consensus.cc:482] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:55:31.382169 6826 raft_consensus.cc:3037] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:31.383127 6826 raft_consensus.cc:504] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.383248 6826 leader_election.cc:288] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [CANDIDATE]: Term 1 election: Requested vote from peers f44e603a6ab140ae8d452b7e12da1808 (127.12.15.67:38977), 07bf17d898bb457889367a72562255de (127.12.15.65:39319)
I20260806 03:55:31.383455 6316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb89d0e5bea4e4fbf847e1c95f1f0b7" candidate_uuid: "2b4d845966664b719aba495f4c97ab8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "07bf17d898bb457889367a72562255de"
I20260806 03:55:31.383482 6727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb89d0e5bea4e4fbf847e1c95f1f0b7" candidate_uuid: "2b4d845966664b719aba495f4c97ab8c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44e603a6ab140ae8d452b7e12da1808"
I20260806 03:55:31.383528 6316 raft_consensus.cc:3037] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:31.383563 6727 raft_consensus.cc:3037] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:31.384385 6727 raft_consensus.cc:2454] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b4d845966664b719aba495f4c97ab8c in term 1.
I20260806 03:55:31.384385 6316 raft_consensus.cc:2454] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b4d845966664b719aba495f4c97ab8c in term 1.
I20260806 03:55:31.384588 6422 leader_election.cc:302] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 07bf17d898bb457889367a72562255de, 2b4d845966664b719aba495f4c97ab8c; no voters:
I20260806 03:55:31.384708 6826 raft_consensus.cc:2781] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:31.384843 6826 raft_consensus.cc:686] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 1 LEADER]: Becoming Leader. State: Replica: 2b4d845966664b719aba495f4c97ab8c, State: Running, Role: LEADER
I20260806 03:55:31.384933 6826 consensus_queue.cc:237] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } }
I20260806 03:55:31.385143 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.382073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:36501 (local address 127.12.15.66:39857)
0806 03:55:31.382146 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.382148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.382280 (+ 132us) server_negotiation.cc:407] Connection header received
0806 03:55:31.382301 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.382301 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.382311 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.382351 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:31.382508 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.382973 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.384282 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.384779 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.384820 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.384826 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.385054 (+ 228us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.385089 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260806 03:55:31.385162 6838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.381912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:39857 (local address 127.12.15.65:36501)
0806 03:55:31.382192 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.382200 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.382254 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.382364 (+ 110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.382364 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.382366 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:31.382425 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.382427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.383056 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.383056 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.384187 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.384190 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.384882 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.384883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.384977 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.384980 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.385041 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.385079 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":10,"thread_start_us":52,"threads_started":1}
I20260806 03:55:31.385392 6816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.382021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:59883 (local address 127.12.15.67:38977)
0806 03:55:31.382349 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.382351 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.382358 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:55:31.382376 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.382377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.382384 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.382432 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:31.382593 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.383215 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.384439 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.384939 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.384977 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.384983 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.385305 (+ 322us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.385340 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307}
I20260806 03:55:31.385411 6837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.381953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38977 (local address 127.12.15.65:59883)
0806 03:55:31.382209 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.382219 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.382271 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.382437 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.382438 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.382439 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:31.382497 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.382499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.383307 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.383308 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.384356 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.384358 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.385105 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.385106 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.385236 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.385241 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.385299 (+ 58us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.385335 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":77,"threads_started":1}
I20260806 03:55:31.385432 6527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb89d0e5bea4e4fbf847e1c95f1f0b7" candidate_uuid: "07bf17d898bb457889367a72562255de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b4d845966664b719aba495f4c97ab8c" is_pre_election: true
I20260806 03:55:31.385785 6094 catalog_manager.cc:5495] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b4d845966664b719aba495f4c97ab8c (127.12.15.66). New cstate: current_term: 1 leader_uuid: "2b4d845966664b719aba495f4c97ab8c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b4d845966664b719aba495f4c97ab8c" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 39857 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 } health_report { overall_health: UNKNOWN } } }
I20260806 03:55:31.386104 6727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9eb89d0e5bea4e4fbf847e1c95f1f0b7" candidate_uuid: "07bf17d898bb457889367a72562255de" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f44e603a6ab140ae8d452b7e12da1808" is_pre_election: true
I20260806 03:55:31.386216 6727 raft_consensus.cc:2379] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 07bf17d898bb457889367a72562255de in current term 1: Already voted for candidate 2b4d845966664b719aba495f4c97ab8c in this term.
I20260806 03:55:31.386427 6217 leader_election.cc:302] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 07bf17d898bb457889367a72562255de; no voters: 2b4d845966664b719aba495f4c97ab8c, f44e603a6ab140ae8d452b7e12da1808
I20260806 03:55:31.386540 6830 raft_consensus.cc:2726] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 03:55:31.469619 6814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.467414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51442 (local address 127.12.15.66:39857)
0806 03:55:31.467462 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.467464 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.467531 (+ 67us) server_negotiation.cc:407] Connection header received
0806 03:55:31.467616 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.467617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.467639 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.467687 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:55:31.467790 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.468260 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.469011 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.469197 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.469250 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.469350 (+ 100us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:55:31.469453 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:55:31.469516 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.469542 (+ 26us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.469557 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:55:31.471474 6727 raft_consensus.cc:1261] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 1 FOLLOWER]: Refusing update from remote peer 2b4d845966664b719aba495f4c97ab8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:31.471558 6316 raft_consensus.cc:1261] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 1 FOLLOWER]: Refusing update from remote peer 2b4d845966664b719aba495f4c97ab8c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:31.471863 6839 consensus_queue.cc:1035] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "07bf17d898bb457889367a72562255de" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 39319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:31.471985 6826 consensus_queue.cc:1035] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f44e603a6ab140ae8d452b7e12da1808" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:31.475399 6846 mvcc.cc:205] Tried to move back new op lower bound from 7315409024904470528 to 7315409024553136128. Current Snapshot: MvccSnapshot[applied={T|T < 7315409024904470528 or (T in {7315409024904470528})}]
I20260806 03:55:31.477532 6094 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:31.477635 6094 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:31.478339 6094 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 86b1bd063c66460786d9a9bce08d5143: Sending DeleteTablet for 3 replicas of tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7
I20260806 03:55:31.478657 6497 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:31 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:51426
I20260806 03:55:31.478696 6700 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:31 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:56944
I20260806 03:55:31.478917 6283 tablet_service.cc:1514] Processing DeleteTablet for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:31 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:38178
I20260806 03:55:31.479055 6855 tablet_replica.cc:330] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: stopping tablet replica
I20260806 03:55:31.479163 6856 tablet_replica.cc:330] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de: stopping tablet replica
I20260806 03:55:31.479152 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 6191
I20260806 03:55:31.479198 6855 raft_consensus.cc:2229] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:31.479296 6856 raft_consensus.cc:2229] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:31.479349 6855 raft_consensus.cc:2258] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:31.479442 6856 raft_consensus.cc:2258] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 07bf17d898bb457889367a72562255de [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:31.479781 6854 tablet_replica.cc:330] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: stopping tablet replica
I20260806 03:55:31.480041 6855 ts_tablet_manager.cc:1905] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:31.480041 6854 raft_consensus.cc:2229] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:55:31.480168 6854 raft_consensus.cc:2258] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:31.481575 6855 ts_tablet_manager.cc:1918] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:31.481665 6855 log.cc:1196] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/9eb89d0e5bea4e4fbf847e1c95f1f0b7
I20260806 03:55:31.481885 6855 ts_tablet_manager.cc:1939] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P f44e603a6ab140ae8d452b7e12da1808: Deleting consensus metadata
I20260806 03:55:31.482327 6067 catalog_manager.cc:4841] TS f44e603a6ab140ae8d452b7e12da1808 (127.12.15.67:38977): tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7 (table test-table [id=0fa201667b6c46f2ad1d8e4e7d85cc4a]) successfully deleted
I20260806 03:55:31.484311 6854 ts_tablet_manager.cc:1905] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:31.485687 6854 ts_tablet_manager.cc:1918] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:31.485759 6854 log.cc:1196] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/9eb89d0e5bea4e4fbf847e1c95f1f0b7
I20260806 03:55:31.485998 6854 ts_tablet_manager.cc:1939] T 9eb89d0e5bea4e4fbf847e1c95f1f0b7 P 2b4d845966664b719aba495f4c97ab8c: Deleting consensus metadata
I20260806 03:55:31.486402 6070 catalog_manager.cc:4841] TS 2b4d845966664b719aba495f4c97ab8c (127.12.15.66:39857): tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7 (table test-table [id=0fa201667b6c46f2ad1d8e4e7d85cc4a]) successfully deleted
W20260806 03:55:31.507892 6068 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:39319 (error 108)
I20260806 03:55:31.507931 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 6391
I20260806 03:55:31.510028 6810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.509853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:39319 (local address 127.0.0.1:38192)
0806 03:55:31.509935 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.509988 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:39319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260806 03:55:31.511673 6068 catalog_manager.cc:4584] TS 07bf17d898bb457889367a72562255de (127.12.15.65:39319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9eb89d0e5bea4e4fbf847e1c95f1f0b7: Network error: Client connection negotiation failed: client connection to 127.12.15.65:39319: connect: Connection refused (error 111)
I20260806 03:55:31.514943 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 6601
I20260806 03:55:31.524901 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 6043
2026-08-06T03:55:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (3171 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-06T03:55:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:31Z Disabled control of system clock
I20260806 03:55:31.556625 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:43291
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:33761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:43291
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:31.636879 6862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:31.637104 6862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:31.637123 6862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:31.638233 6862 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:31.638271 6862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:31.638283 6862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:31.638293 6862 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:31.638303 6862 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:31.639599 6862 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:43291
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:43291
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:31.639814 6862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:31.640089 6862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:31.644763 6873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:31.644737 6877 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:31.644779 6872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:31.644837 6862 server_base.cc:1034] running on GCE node
I20260806 03:55:31.645179 6862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:31.645465 6862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:31.646605 6862 hybrid_clock.cc:648] HybridClock initialized: now 1785988531646585 us; error 37 us; skew 500 ppm
I20260806 03:55:31.647809 6862 webserver.cc:460] Webserver started at http://127.12.15.126:41217/ using document root <none> and password file <none>
I20260806 03:55:31.648033 6862 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:31.648078 6862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:31.648197 6862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:31.649202 6862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "7519b261cd56d8a753a265ba6f71a8f0"
server_key_iv: "adbe31b102383de12b505e467d5e67a1"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.649524 6862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "7519b261cd56d8a753a265ba6f71a8f0"
server_key_iv: "adbe31b102383de12b505e467d5e67a1"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.650933 6862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:55:31.651594 6892 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:31.651764 6862 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:31.651849 6862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "7519b261cd56d8a753a265ba6f71a8f0"
server_key_iv: "adbe31b102383de12b505e467d5e67a1"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.651906 6862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:31.675401 6862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:31.675707 6862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:31.675840 6862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:31.679591 6862 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:43291
I20260806 03:55:31.679631 6971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:43291 every 8 connection(s)
I20260806 03:55:31.679937 6862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:31.680896 6972 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:31.680923 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 6862
I20260806 03:55:31.681016 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:31.681205 12349 external_mini_cluster.cc:1428] Setting key 5f33984be77cf28d79884f90455b82da
I20260806 03:55:31.684331 6972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e: Bootstrap starting.
I20260806 03:55:31.684918 6972 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:31.685192 6972 log.cc:824] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:31.685846 6972 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e: No bootstrap required, opened a new log
I20260806 03:55:31.686856 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.681697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58740 (local address 127.12.15.126:43291)
0806 03:55:31.681930 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.681940 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.681973 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:55:31.682453 (+ 480us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.682464 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.682550 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.682700 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:31.682860 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.684179 (+ 1319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.685011 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.685452 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.685566 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.686241 (+ 675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:31.686245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:31.686246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:31.686398 (+ 152us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:31.686431 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:31.686441 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:31.686496 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.686580 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.686641 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":102,"threads_started":1}
I20260806 03:55:31.687908 6972 raft_consensus.cc:348] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43291 } }
I20260806 03:55:31.688037 6972 raft_consensus.cc:374] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:31.688069 6972 raft_consensus.cc:729] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bcc3d7ec7bb04deab142fa6d612dbf2e, State: Initialized, Role: FOLLOWER
I20260806 03:55:31.688180 6972 consensus_queue.cc:260] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43291 } }
I20260806 03:55:31.688241 6972 raft_consensus.cc:388] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:31.688270 6972 raft_consensus.cc:482] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:31.688318 6972 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:31.688880 6972 raft_consensus.cc:504] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43291 } }
I20260806 03:55:31.688995 6972 leader_election.cc:302] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bcc3d7ec7bb04deab142fa6d612dbf2e; no voters:
I20260806 03:55:31.689138 6972 leader_election.cc:288] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:31.689183 6980 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:31.689361 6980 raft_consensus.cc:686] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [term 1 LEADER]: Becoming Leader. State: Replica: bcc3d7ec7bb04deab142fa6d612dbf2e, State: Running, Role: LEADER
I20260806 03:55:31.689365 6972 sys_catalog.cc:564] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:31.689520 6980 consensus_queue.cc:237] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43291 } }
I20260806 03:55:31.689877 6982 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader bcc3d7ec7bb04deab142fa6d612dbf2e. Latest consensus state: current_term: 1 leader_uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43291 } } }
I20260806 03:55:31.689975 6982 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:31.689877 6981 sys_catalog.cc:455] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bcc3d7ec7bb04deab142fa6d612dbf2e" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 43291 } } }
I20260806 03:55:31.690140 6981 sys_catalog.cc:458] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:31.690276 6992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:31.690771 6992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:31.692137 6992 catalog_manager.cc:1347] Generated new cluster ID: ab24744a1a8e412a8668e827b833d9da
I20260806 03:55:31.692188 6992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:31.716925 6992 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:31.717411 6992 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:31.724644 6992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e: Generated new TSK 0
I20260806 03:55:31.724812 6992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:31.732949 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43291
--builtin_ntp_servers=127.12.15.84:33761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:31.842547 7011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:31.842749 7011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:31.842779 7011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:31.843878 7011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:31.843916 7011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:31.843948 7011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:55:31.845271 7011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7011
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:31.845502 7011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:31.845801 7011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:31.850768 7020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:31.850778 7023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:31.850807 7019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:31.850893 7011 server_base.cc:1034] running on GCE node
I20260806 03:55:31.851207 7011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:31.851424 7011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:31.852583 7011 hybrid_clock.cc:648] HybridClock initialized: now 1785988531852564 us; error 33 us; skew 500 ppm
I20260806 03:55:31.853797 7011 webserver.cc:460] Webserver started at http://127.12.15.65:45967/ using document root <none> and password file <none>
I20260806 03:55:31.854043 7011 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:31.854089 7011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:31.854238 7011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:31.855329 7011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "db89ce1a8cc34e548a42f4c62170228d"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "849a0c60484f0260122a245bae0e1c7b"
server_key_iv: "bdc78ec367359b2addfb6ff61db6a579"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.855649 7011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "db89ce1a8cc34e548a42f4c62170228d"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "849a0c60484f0260122a245bae0e1c7b"
server_key_iv: "bdc78ec367359b2addfb6ff61db6a579"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.856765 7011 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:55:31.857402 7032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:31.857566 7011 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:31.857648 7011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "db89ce1a8cc34e548a42f4c62170228d"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "849a0c60484f0260122a245bae0e1c7b"
server_key_iv: "bdc78ec367359b2addfb6ff61db6a579"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.857710 7011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:31.867847 7011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:31.868089 7011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:31.868204 7011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:31.868386 7011 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:31.868695 7011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:31.868729 7011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:31.868757 7011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:31.868779 7011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:31.874330 7011 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:44051
I20260806 03:55:31.874388 7206 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:44051 every 8 connection(s)
I20260806 03:55:31.874680 7011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:55:31.879336 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.875347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:50939 (local address 127.12.15.126:43291)
0806 03:55:31.875396 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.875399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.875613 (+ 214us) server_negotiation.cc:407] Connection header received
0806 03:55:31.876069 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.876070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.876084 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.876132 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:31.876773 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.877273 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.878187 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.878373 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.878425 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.879119 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:31.879120 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:31.879121 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:31.879125 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:31.879140 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:31.879144 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:31.879183 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.879253 (+ 70us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.879277 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:55:31.879535 7213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.875334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:43291 (local address 127.12.15.65:50939)
0806 03:55:31.875507 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.875539 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.875991 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.876189 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.876192 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.876200 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:31.876687 (+ 487us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.876692 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.877337 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.877338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.878102 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.878104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.878438 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.878438 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.878550 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.878998 (+ 448us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:31.878998 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:31.879025 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:31.879046 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:31.879189 (+ 143us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:31.879191 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:31.879194 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.879248 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.879328 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":81,"threads_started":1}
I20260806 03:55:31.879704 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 7011
I20260806 03:55:31.879799 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:55:31.879976 12349 external_mini_cluster.cc:1428] Setting key aeb0264a6265284a38000e7184243651
I20260806 03:55:31.879978 7208 heartbeater.cc:347] Connected to a master server at 127.12.15.126:43291
I20260806 03:55:31.880060 7208 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:31.880262 7208 heartbeater.cc:511] Master 127.12.15.126:43291 requested a full tablet report, sending...
I20260806 03:55:31.880796 6916 ts_manager.cc:194] Registered new tserver with Master: db89ce1a8cc34e548a42f4c62170228d (127.12.15.65:44051)
I20260806 03:55:31.880846 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43291
--builtin_ntp_servers=127.12.15.84:33761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 03:55:31.881394 6916 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.65:50939
W20260806 03:55:31.961401 7215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:31.961611 7215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:31.961640 7215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:31.962759 7215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:31.962798 7215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:31.962831 7215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:55:31.964111 7215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:31.964329 7215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:31.964591 7215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:31.969408 7223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:31.969442 7226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:31.969414 7222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:31.969439 7215 server_base.cc:1034] running on GCE node
I20260806 03:55:31.969825 7215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:31.970068 7215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:31.971210 7215 hybrid_clock.cc:648] HybridClock initialized: now 1785988531971196 us; error 29 us; skew 500 ppm
I20260806 03:55:31.972440 7215 webserver.cc:460] Webserver started at http://127.12.15.66:43831/ using document root <none> and password file <none>
I20260806 03:55:31.972644 7215 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:31.972692 7215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:31.972812 7215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:31.973809 7215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "368a7919e4ab4715a11bef16b5876e9d"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "9e00e1cce6dbf7c9448858227cfd33a3"
server_key_iv: "0138267aabb9a9a87ab94005602bf5a1"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.974159 7215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "368a7919e4ab4715a11bef16b5876e9d"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "9e00e1cce6dbf7c9448858227cfd33a3"
server_key_iv: "0138267aabb9a9a87ab94005602bf5a1"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.975464 7215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:55:31.976146 7237 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:31.976326 7215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:31.976418 7215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "368a7919e4ab4715a11bef16b5876e9d"
format_stamp: "Formatted at 2026-08-06 03:55:31 on dist-test-slave-bqcl"
server_key: "9e00e1cce6dbf7c9448858227cfd33a3"
server_key_iv: "0138267aabb9a9a87ab94005602bf5a1"
server_key_version: "encryptionkey@0"
I20260806 03:55:31.976485 7215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:31.983079 7215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:31.983332 7215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:31.983453 7215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:31.983651 7215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:31.983954 7215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:31.983989 7215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:31.984016 7215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:31.984048 7215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:31.989603 7215 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:45011
I20260806 03:55:31.989667 7411 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:45011 every 8 connection(s)
I20260806 03:55:31.989974 7215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:55:31.994953 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.990705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:44461 (local address 127.12.15.126:43291)
0806 03:55:31.990746 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:31.990748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:31.990984 (+ 236us) server_negotiation.cc:407] Connection header received
0806 03:55:31.991521 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:31.991522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:31.991534 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:31.991592 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:31.992224 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.992722 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.993691 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.993882 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.993934 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.994747 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:31.994748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:31.994749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:31.994753 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:31.994768 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:31.994771 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:31.994814 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:31.994870 (+ 56us) server_negotiation.cc:299] Negotiation successful
0806 03:55:31.994893 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:55:31.995160 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:31.990702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:43291 (local address 127.12.15.66:44461)
0806 03:55:31.990863 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:31.990898 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:31.991402 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:31.991649 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:31.991652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:31.991661 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:31.992160 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.992165 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.992793 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.992793 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.993618 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:31.993620 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:31.993943 (+ 323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:31.993944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:31.994064 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:31.994623 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:31.994623 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:31.994652 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:31.994675 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:31.994816 (+ 141us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:31.994818 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:31.994820 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:55:31.994867 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:55:31.994945 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":72,"threads_started":1}
I20260806 03:55:31.995669 7419 heartbeater.cc:347] Connected to a master server at 127.12.15.126:43291
I20260806 03:55:31.995760 7419 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:31.995913 7419 heartbeater.cc:511] Master 127.12.15.126:43291 requested a full tablet report, sending...
I20260806 03:55:31.996212 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 7215
I20260806 03:55:31.996279 6916 ts_manager.cc:194] Registered new tserver with Master: 368a7919e4ab4715a11bef16b5876e9d (127.12.15.66:45011)
I20260806 03:55:31.996310 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:55:31.996482 12349 external_mini_cluster.cc:1428] Setting key b42acbe6ccf1dde36ea2720856d71989
I20260806 03:55:31.996680 6916 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.66:44461
I20260806 03:55:31.997488 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43291
--builtin_ntp_servers=127.12.15.84:33761
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:32.079634 7422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:32.079846 7422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:32.079870 7422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:32.081007 7422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:32.081055 7422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:32.081081 7422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:55:32.082406 7422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:33761
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:43291
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7422
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:32.082653 7422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:32.082939 7422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:32.087755 7433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:32.087766 7428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:32.087774 7430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:32.088049 7422 server_base.cc:1034] running on GCE node
I20260806 03:55:32.088328 7422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:32.088564 7422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:32.089740 7422 hybrid_clock.cc:648] HybridClock initialized: now 1785988532089726 us; error 24 us; skew 500 ppm
I20260806 03:55:32.091014 7422 webserver.cc:460] Webserver started at http://127.12.15.67:32995/ using document root <none> and password file <none>
I20260806 03:55:32.091223 7422 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:32.091272 7422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:32.091400 7422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:32.092411 7422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "56e52e09dba34101ad899490baec4e86"
format_stamp: "Formatted at 2026-08-06 03:55:32 on dist-test-slave-bqcl"
server_key: "a8ba110000ebdb7adb2b7c46c6a503e0"
server_key_iv: "1cf1aa4fb3d0632ed6793d0077f84915"
server_key_version: "encryptionkey@0"
I20260806 03:55:32.092744 7422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "56e52e09dba34101ad899490baec4e86"
format_stamp: "Formatted at 2026-08-06 03:55:32 on dist-test-slave-bqcl"
server_key: "a8ba110000ebdb7adb2b7c46c6a503e0"
server_key_iv: "1cf1aa4fb3d0632ed6793d0077f84915"
server_key_version: "encryptionkey@0"
I20260806 03:55:32.093983 7422 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:32.094781 7442 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:32.094940 7422 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:32.095034 7422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "56e52e09dba34101ad899490baec4e86"
format_stamp: "Formatted at 2026-08-06 03:55:32 on dist-test-slave-bqcl"
server_key: "a8ba110000ebdb7adb2b7c46c6a503e0"
server_key_iv: "1cf1aa4fb3d0632ed6793d0077f84915"
server_key_version: "encryptionkey@0"
I20260806 03:55:32.095094 7422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:32.113897 7422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:32.114195 7422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:32.114328 7422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:32.114540 7422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:32.114858 7422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:32.114893 7422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:32.114923 7422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:32.114948 7422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:32.120579 7422 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:38941
I20260806 03:55:32.120646 7627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:38941 every 8 connection(s)
I20260806 03:55:32.120946 7422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:55:32.122972 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 7422
I20260806 03:55:32.123065 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:55:32.123267 12349 external_mini_cluster.cc:1428] Setting key 82903b2a2ac1f150f101566cec8f29ca
I20260806 03:55:32.125976 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.121634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:54767 (local address 127.12.15.126:43291)
0806 03:55:32.121678 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:32.121680 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:32.121888 (+ 208us) server_negotiation.cc:407] Connection header received
0806 03:55:32.122433 (+ 545us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:32.122434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:32.122444 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:32.122481 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:32.123173 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.123828 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.124821 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.125018 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.125075 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.125782 (+ 707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:32.125783 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:32.125784 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:32.125788 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:32.125802 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:32.125805 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:32.125842 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:32.125897 (+ 55us) server_negotiation.cc:299] Negotiation successful
0806 03:55:32.125918 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:55:32.126215 7632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.121593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:43291 (local address 127.12.15.67:54767)
0806 03:55:32.121772 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:32.121813 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:32.122329 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:32.122544 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:32.122547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:32.122555 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:32.123080 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.123087 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.123916 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.123917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.124745 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.124748 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.125086 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.125086 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.125205 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.125665 (+ 460us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:32.125666 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:32.125694 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:32.125716 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:32.125844 (+ 128us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:32.125846 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:32.125849 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:32.125891 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 03:55:32.125966 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260806 03:55:32.126593 7628 heartbeater.cc:347] Connected to a master server at 127.12.15.126:43291
I20260806 03:55:32.126685 7628 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:32.126856 7628 heartbeater.cc:511] Master 127.12.15.126:43291 requested a full tablet report, sending...
I20260806 03:55:32.127170 6911 ts_manager.cc:194] Registered new tserver with Master: 56e52e09dba34101ad899490baec4e86 (127.12.15.67:38941)
I20260806 03:55:32.127496 6911 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.67:54767
I20260806 03:55:32.134879 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:55:32.138569 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.136168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58754 (local address 127.12.15.126:43291)
0806 03:55:32.136214 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:32.136216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:32.136304 (+ 88us) server_negotiation.cc:407] Connection header received
0806 03:55:32.136356 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:32.136357 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:32.136365 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:32.136400 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:32.136533 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.136979 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.137653 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.137818 (+ 165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.137859 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.138408 (+ 549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:32.138409 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:32.138409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:32.138413 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:32.138425 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:32.138428 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:32.138466 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:32.138497 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 03:55:32.138516 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:55:32.139966 6911 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58754:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:55:32.141475 6911 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:55:32.147365 7642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.143496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:45011 (local address 127.0.0.1:48906)
0806 03:55:32.143679 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:32.143722 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:32.143786 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:32.144009 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:32.144012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:32.144014 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:32.144075 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.144077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.145079 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.145080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.146221 (+ 1141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.146224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.147082 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.147083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.147165 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.147179 (+ 14us) client_negotiation.cc:769] Sending connection context
0806 03:55:32.147241 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:55:32.147291 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":57,"threads_started":1}
I20260806 03:55:32.147377 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.143691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48906 (local address 127.12.15.66:45011)
0806 03:55:32.143864 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:32.143868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:32.143884 (+ 16us) server_negotiation.cc:407] Connection header received
0806 03:55:32.143921 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:32.143923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:32.143935 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:32.143998 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:32.144169 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.145010 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.146311 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.147012 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.147070 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.147163 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:32.147253 (+ 90us) server_negotiation.cc:299] Negotiation successful
0806 03:55:32.147291 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":83,"threads_started":1}
I20260806 03:55:32.147675 7649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.143705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44051 (local address 127.0.0.1:48960)
0806 03:55:32.144098 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:32.144107 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:32.144146 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:32.144488 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:32.144489 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:32.144491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:32.144528 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.144530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.145713 (+ 1183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.145714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.146571 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.146574 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.147432 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.147434 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.147493 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.147498 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:55:32.147566 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 03:55:32.147605 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":69,"threads_started":1}
I20260806 03:55:32.147708 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.143925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.12.15.65:44051)
0806 03:55:32.144337 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:32.144341 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:32.144351 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:55:32.144387 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:32.144390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:32.144405 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:32.144477 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:32.144596 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.145645 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.146642 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.147250 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.147314 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.147395 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:32.147586 (+ 191us) server_negotiation.cc:299] Negotiation successful
0806 03:55:32.147629 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":98,"threads_started":1}
I20260806 03:55:32.147888 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.143496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38941 (local address 127.0.0.1:33264)
0806 03:55:32.143869 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:32.143879 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:32.143927 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:32.144248 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:32.144249 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:32.144251 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:32.144294 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.144295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.145528 (+ 1233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.145529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.146781 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.146788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.147706 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.147707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.147766 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.147769 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:32.147811 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 03:55:32.147845 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":62,"thread_start_us":84,"threads_started":1}
I20260806 03:55:32.148146 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.143581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33264 (local address 127.12.15.67:38941)
0806 03:55:32.144050 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:32.144055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:32.144064 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:55:32.144100 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:32.144102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:32.144117 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:32.144186 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:32.144368 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.145444 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.146863 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.147541 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.147597 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.147674 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:32.148033 (+ 359us) server_negotiation.cc:299] Negotiation successful
0806 03:55:32.148074 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":108,"threads_started":1}
I20260806 03:55:32.148315 7109 tablet_service.cc:1467] Processing CreateTablet for tablet 5ebf2e738e2941a381aadcdad9983504 (DEFAULT_TABLE table=test-table [id=3218bc300b684afe83f383f959933bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:32.148377 7309 tablet_service.cc:1467] Processing CreateTablet for tablet 5ebf2e738e2941a381aadcdad9983504 (DEFAULT_TABLE table=test-table [id=3218bc300b684afe83f383f959933bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:32.148687 7109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ebf2e738e2941a381aadcdad9983504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:32.148751 7309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ebf2e738e2941a381aadcdad9983504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:32.148769 7522 tablet_service.cc:1467] Processing CreateTablet for tablet 5ebf2e738e2941a381aadcdad9983504 (DEFAULT_TABLE table=test-table [id=3218bc300b684afe83f383f959933bcb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:32.149101 7522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5ebf2e738e2941a381aadcdad9983504. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:32.151260 7652 tablet_bootstrap.cc:492] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d: Bootstrap starting.
I20260806 03:55:32.151932 7652 tablet_bootstrap.cc:654] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:32.152079 7654 tablet_bootstrap.cc:492] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Bootstrap starting.
I20260806 03:55:32.152408 7652 log.cc:824] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:32.152534 7656 tablet_bootstrap.cc:492] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: Bootstrap starting.
I20260806 03:55:32.152740 7654 tablet_bootstrap.cc:654] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:32.153049 7654 log.cc:824] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:32.153076 7656 tablet_bootstrap.cc:654] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:32.153168 7652 tablet_bootstrap.cc:492] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d: No bootstrap required, opened a new log
I20260806 03:55:32.153239 7652 ts_tablet_manager.cc:1397] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:32.153365 7656 log.cc:824] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:32.153903 7654 tablet_bootstrap.cc:492] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: No bootstrap required, opened a new log
I20260806 03:55:32.153973 7654 ts_tablet_manager.cc:1397] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:32.154050 7656 tablet_bootstrap.cc:492] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: No bootstrap required, opened a new log
I20260806 03:55:32.154115 7656 ts_tablet_manager.cc:1397] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:32.155723 7652 raft_consensus.cc:348] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.155900 7652 raft_consensus.cc:374] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:32.155936 7652 raft_consensus.cc:729] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db89ce1a8cc34e548a42f4c62170228d, State: Initialized, Role: FOLLOWER
I20260806 03:55:32.156052 7652 consensus_queue.cc:260] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.156087 7656 raft_consensus.cc:348] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.156225 7656 raft_consensus.cc:374] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:32.156262 7656 raft_consensus.cc:729] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56e52e09dba34101ad899490baec4e86, State: Initialized, Role: FOLLOWER
I20260806 03:55:32.156328 7652 ts_tablet_manager.cc:1428] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:55:32.156399 7656 consensus_queue.cc:260] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.156486 7208 heartbeater.cc:503] Master 127.12.15.126:43291 was elected leader, sending a full tablet report...
I20260806 03:55:32.156450 7654 raft_consensus.cc:348] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.156605 7654 raft_consensus.cc:374] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:32.156648 7656 ts_tablet_manager.cc:1428] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:55:32.156646 7654 raft_consensus.cc:729] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 368a7919e4ab4715a11bef16b5876e9d, State: Initialized, Role: FOLLOWER
I20260806 03:55:32.156754 7628 heartbeater.cc:503] Master 127.12.15.126:43291 was elected leader, sending a full tablet report...
I20260806 03:55:32.156764 7654 consensus_queue.cc:260] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.157002 7654 ts_tablet_manager.cc:1428] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:55:32.157027 7419 heartbeater.cc:503] Master 127.12.15.126:43291 was elected leader, sending a full tablet report...
I20260806 03:55:32.235505 7669 raft_consensus.cc:482] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:32.235663 7669 raft_consensus.cc:504] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.236110 7669 leader_election.cc:288] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 56e52e09dba34101ad899490baec4e86 (127.12.15.67:38941), db89ce1a8cc34e548a42f4c62170228d (127.12.15.65:44051)
I20260806 03:55:32.240038 7644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.236399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:40467 (local address 127.12.15.67:38941)
0806 03:55:32.236449 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:32.236451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:32.236460 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:55:32.236489 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:32.236490 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:32.236500 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:32.236554 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:32.236738 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.237550 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.239025 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.239592 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.239666 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.239675 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:32.239895 (+ 220us) server_negotiation.cc:299] Negotiation successful
0806 03:55:32.239949 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260806 03:55:32.240048 7670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.236421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44051 (local address 127.12.15.66:50715)
0806 03:55:32.236621 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:32.236633 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:32.236698 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:32.236820 (+ 122us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:32.236820 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:32.236821 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:32.236878 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.236879 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.237691 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.237692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.238916 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.238918 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.239694 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.239694 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.239787 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.239790 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:32.239858 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 03:55:32.239907 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":11,"thread_start_us":120,"threads_started":1}
I20260806 03:55:32.240159 7651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.236554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:50715 (local address 127.12.15.65:44051)
0806 03:55:32.236620 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:32.236621 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:32.236693 (+ 72us) server_negotiation.cc:407] Connection header received
0806 03:55:32.236758 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:32.236758 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:32.236769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:32.236818 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:32.236923 (+ 105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.237590 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.239016 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.239592 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.239666 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.239675 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:32.240042 (+ 367us) server_negotiation.cc:299] Negotiation successful
0806 03:55:32.240093 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:55:32.240054 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.236293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38941 (local address 127.12.15.66:40467)
0806 03:55:32.236352 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:32.236366 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:32.236426 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:32.236573 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:32.236574 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:32.236577 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:32.236658 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.236661 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.237636 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.237637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.238915 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:32.238918 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.239693 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.239694 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:32.239784 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.239788 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:32.239858 (+ 70us) client_negotiation.cc:240] Negotiation successful
0806 03:55:32.239910 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":10}
W20260806 03:55:32.240525 7420 tablet.cc:2369] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:55:32.240484 7137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ebf2e738e2941a381aadcdad9983504" candidate_uuid: "368a7919e4ab4715a11bef16b5876e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db89ce1a8cc34e548a42f4c62170228d" is_pre_election: true
I20260806 03:55:32.240600 7553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ebf2e738e2941a381aadcdad9983504" candidate_uuid: "368a7919e4ab4715a11bef16b5876e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e52e09dba34101ad899490baec4e86" is_pre_election: true
I20260806 03:55:32.240657 7137 raft_consensus.cc:2454] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 368a7919e4ab4715a11bef16b5876e9d in term 0.
I20260806 03:55:32.240737 7553 raft_consensus.cc:2454] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 368a7919e4ab4715a11bef16b5876e9d in term 0.
I20260806 03:55:32.240891 7242 leader_election.cc:302] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 368a7919e4ab4715a11bef16b5876e9d, db89ce1a8cc34e548a42f4c62170228d; no voters:
I20260806 03:55:32.241034 7669 raft_consensus.cc:2781] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:55:32.241096 7669 raft_consensus.cc:482] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:55:32.241132 7669 raft_consensus.cc:3037] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:32.241925 7669 raft_consensus.cc:504] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.242053 7669 leader_election.cc:288] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [CANDIDATE]: Term 1 election: Requested vote from peers 56e52e09dba34101ad899490baec4e86 (127.12.15.67:38941), db89ce1a8cc34e548a42f4c62170228d (127.12.15.65:44051)
I20260806 03:55:32.242283 7553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ebf2e738e2941a381aadcdad9983504" candidate_uuid: "368a7919e4ab4715a11bef16b5876e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "56e52e09dba34101ad899490baec4e86"
I20260806 03:55:32.242300 7137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5ebf2e738e2941a381aadcdad9983504" candidate_uuid: "368a7919e4ab4715a11bef16b5876e9d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db89ce1a8cc34e548a42f4c62170228d"
I20260806 03:55:32.242367 7553 raft_consensus.cc:3037] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:32.242369 7137 raft_consensus.cc:3037] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:32.243052 7553 raft_consensus.cc:2454] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 368a7919e4ab4715a11bef16b5876e9d in term 1.
I20260806 03:55:32.243163 7137 raft_consensus.cc:2454] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 368a7919e4ab4715a11bef16b5876e9d in term 1.
I20260806 03:55:32.243214 7242 leader_election.cc:302] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 368a7919e4ab4715a11bef16b5876e9d, 56e52e09dba34101ad899490baec4e86; no voters:
I20260806 03:55:32.243314 7669 raft_consensus.cc:2781] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:32.243480 7669 raft_consensus.cc:686] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 1 LEADER]: Becoming Leader. State: Replica: 368a7919e4ab4715a11bef16b5876e9d, State: Running, Role: LEADER
I20260806 03:55:32.243585 7669 consensus_queue.cc:237] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } }
I20260806 03:55:32.244428 6916 catalog_manager.cc:5495] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d reported cstate change: term changed from 0 to 1, leader changed from <none> to 368a7919e4ab4715a11bef16b5876e9d (127.12.15.66). New cstate: current_term: 1 leader_uuid: "368a7919e4ab4715a11bef16b5876e9d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "368a7919e4ab4715a11bef16b5876e9d" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 45011 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 } health_report { overall_health: UNKNOWN } } }
I20260806 03:55:32.264941 7650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.262840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48920 (local address 127.12.15.66:45011)
0806 03:55:32.262888 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:32.262890 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:32.262900 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:55:32.262939 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:32.262939 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:32.262956 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:32.263017 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:55:32.263113 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.263592 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.264306 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:32.264512 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:32.264569 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:32.264659 (+ 90us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:55:32.264755 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:55:32.264809 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:32.264821 (+ 12us) server_negotiation.cc:299] Negotiation successful
0806 03:55:32.264835 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:55:32.266687 7553 raft_consensus.cc:1261] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 1 FOLLOWER]: Refusing update from remote peer 368a7919e4ab4715a11bef16b5876e9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:32.266723 7137 raft_consensus.cc:1261] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 1 FOLLOWER]: Refusing update from remote peer 368a7919e4ab4715a11bef16b5876e9d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:32.267038 7671 consensus_queue.cc:1035] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "56e52e09dba34101ad899490baec4e86" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:32.267150 7669 consensus_queue.cc:1035] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [LEADER]: Connected to new peer: Peer: permanent_uuid: "db89ce1a8cc34e548a42f4c62170228d" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 44051 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:32.269564 7678 mvcc.cc:205] Tried to move back new op lower bound from 7315409028161728512 to 7315409028070338560. Current Snapshot: MvccSnapshot[applied={T|T < 7315409028161728512}]
I20260806 03:55:32.269740 7680 mvcc.cc:205] Tried to move back new op lower bound from 7315409028161728512 to 7315409028070338560. Current Snapshot: MvccSnapshot[applied={T|T < 7315409028161728512}]
I20260806 03:55:32.271976 6916 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:32.272055 6916 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58754:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:32.272727 6916 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bcc3d7ec7bb04deab142fa6d612dbf2e: Sending DeleteTablet for 3 replicas of tablet 5ebf2e738e2941a381aadcdad9983504
I20260806 03:55:32.273069 7522 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ebf2e738e2941a381aadcdad9983504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:32 UTC) from {username='slave'} at 127.0.0.1:33264
I20260806 03:55:32.273064 7309 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ebf2e738e2941a381aadcdad9983504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:32 UTC) from {username='slave'} at 127.0.0.1:48906
I20260806 03:55:32.273291 7693 tablet_replica.cc:330] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: stopping tablet replica
I20260806 03:55:32.273294 7109 tablet_service.cc:1514] Processing DeleteTablet for tablet 5ebf2e738e2941a381aadcdad9983504 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:32 UTC) from {username='slave'} at 127.0.0.1:48960
I20260806 03:55:32.273393 7693 raft_consensus.cc:2229] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:32.273504 7693 raft_consensus.cc:2258] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:32.273589 7695 tablet_replica.cc:330] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d: stopping tablet replica
I20260806 03:55:32.273729 7695 raft_consensus.cc:2229] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:32.273774 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 7011
I20260806 03:55:32.273907 7695 raft_consensus.cc:2258] T 5ebf2e738e2941a381aadcdad9983504 P db89ce1a8cc34e548a42f4c62170228d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:32.274439 7693 ts_tablet_manager.cc:1905] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:32.276289 7693 ts_tablet_manager.cc:1918] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:32.276379 7693 log.cc:1196] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/5ebf2e738e2941a381aadcdad9983504
I20260806 03:55:32.276640 7693 ts_tablet_manager.cc:1939] T 5ebf2e738e2941a381aadcdad9983504 P 56e52e09dba34101ad899490baec4e86: Deleting consensus metadata
I20260806 03:55:32.277021 7694 tablet_replica.cc:330] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: stopping tablet replica
I20260806 03:55:32.277055 6895 catalog_manager.cc:4841] TS 56e52e09dba34101ad899490baec4e86 (127.12.15.67:38941): tablet 5ebf2e738e2941a381aadcdad9983504 (table test-table [id=3218bc300b684afe83f383f959933bcb]) successfully deleted
I20260806 03:55:32.277132 7694 raft_consensus.cc:2229] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:55:32.277233 7694 raft_consensus.cc:2258] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:32.277653 7694 ts_tablet_manager.cc:1905] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:32.280375 7694 ts_tablet_manager.cc:1918] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:32.280449 7694 log.cc:1196] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/5ebf2e738e2941a381aadcdad9983504
I20260806 03:55:32.280653 7694 ts_tablet_manager.cc:1939] T 5ebf2e738e2941a381aadcdad9983504 P 368a7919e4ab4715a11bef16b5876e9d: Deleting consensus metadata
I20260806 03:55:32.280952 6896 catalog_manager.cc:4841] TS 368a7919e4ab4715a11bef16b5876e9d (127.12.15.66:45011): tablet 5ebf2e738e2941a381aadcdad9983504 (table test-table [id=3218bc300b684afe83f383f959933bcb]) successfully deleted
W20260806 03:55:32.289112 6895 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:44051 (error 108)
I20260806 03:55:32.289615 7640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:32.289468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:44051 (local address 127.0.0.1:48966)
0806 03:55:32.289529 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:32.289577 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:44051: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 03:55:32.289662 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 7215
W20260806 03:55:32.300179 6895 catalog_manager.cc:4584] TS db89ce1a8cc34e548a42f4c62170228d (127.12.15.65:44051): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5ebf2e738e2941a381aadcdad9983504: Network error: Client connection negotiation failed: client connection to 127.12.15.65:44051: connect: Connection refused (error 111)
I20260806 03:55:32.303833 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 7422
I20260806 03:55:32.314860 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 6862
2026-08-06T03:55:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (788 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:32 dist-test-slave-bqcl krb5kdc[7701](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:32 dist-test-slave-bqcl krb5kdc[7701](info): set up 2 sockets
Aug 06 03:55:32 dist-test-slave-bqcl krb5kdc[7701](info): commencing operation
krb5kdc: starting...
W20260806 03:55:33.446584 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:33 dist-test-slave-bqcl krb5kdc[7701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:55:33.562772 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36939
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:36939
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:33.642733 7719 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:33.642936 7719 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:33.642966 7719 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:33.644070 7719 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:33.644112 7719 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:33.644131 7719 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:33.644146 7719 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:33.644163 7719 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:33.645635 7719 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:36939
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7719
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:33.645869 7719 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:33.646086 7719 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:33.650770 7735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:33.650873 7719 server_base.cc:1034] running on GCE node
W20260806 03:55:33.650804 7726 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:33.650969 7725 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:33.651269 7719 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:33.651563 7719 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:33.652700 7719 hybrid_clock.cc:648] HybridClock initialized: now 1785988533652683 us; error 33 us; skew 500 ppm
Aug 06 03:55:33 dist-test-slave-bqcl krb5kdc[7701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988533, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:33.655189 7719 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:55:33.655599 7719 webserver.cc:460] Webserver started at http://127.12.15.126:39177/ using document root <none> and password file <none>
I20260806 03:55:33.655799 7719 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:33.655848 7719 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:33.656042 7719 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:33.657090 7719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f"
format_stamp: "Formatted at 2026-08-06 03:55:33 on dist-test-slave-bqcl"
server_key: "8d0cc576de3e585edc2d79ebcf76a7a3"
server_key_iv: "f0771923e143af1b3dd6fbdf214f57af"
server_key_version: "encryptionkey@0"
I20260806 03:55:33.657415 7719 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f"
format_stamp: "Formatted at 2026-08-06 03:55:33 on dist-test-slave-bqcl"
server_key: "8d0cc576de3e585edc2d79ebcf76a7a3"
server_key_iv: "f0771923e143af1b3dd6fbdf214f57af"
server_key_version: "encryptionkey@0"
I20260806 03:55:33.658726 7719 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:33.659416 7745 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:33.659555 7719 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:33.659642 7719 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f"
format_stamp: "Formatted at 2026-08-06 03:55:33 on dist-test-slave-bqcl"
server_key: "8d0cc576de3e585edc2d79ebcf76a7a3"
server_key_iv: "f0771923e143af1b3dd6fbdf214f57af"
server_key_version: "encryptionkey@0"
I20260806 03:55:33.659713 7719 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:33.679270 7719 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:33.681077 7719 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:33.681210 7719 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:33.684966 7719 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:36939
I20260806 03:55:33.685003 7827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:36939 every 8 connection(s)
I20260806 03:55:33.685328 7719 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:33.686115 7828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:33.687366 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 7719
I20260806 03:55:33.687489 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:33.687696 12349 external_mini_cluster.cc:1428] Setting key a726ef5cf4147274f60753c1e55c8d89
I20260806 03:55:33.688740 7828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f: Bootstrap starting.
I20260806 03:55:33.689296 7828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:33.689556 7828 log.cc:824] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:33.690196 7828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f: No bootstrap required, opened a new log
I20260806 03:55:33.691922 7828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36939 } }
I20260806 03:55:33.692067 7828 raft_consensus.cc:374] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:33.692104 7828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 288e0f1c291f4d92a1e3d1b6a76e1a3f, State: Initialized, Role: FOLLOWER
I20260806 03:55:33.692225 7828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36939 } }
I20260806 03:55:33.692301 7828 raft_consensus.cc:388] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:33.692332 7828 raft_consensus.cc:482] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:33.692384 7828 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:33.693048 7828 raft_consensus.cc:504] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36939 } }
I20260806 03:55:33.693172 7828 leader_election.cc:302] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 288e0f1c291f4d92a1e3d1b6a76e1a3f; no voters:
I20260806 03:55:33.693341 7828 leader_election.cc:288] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:33.693379 7838 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:33.693522 7838 raft_consensus.cc:686] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [term 1 LEADER]: Becoming Leader. State: Replica: 288e0f1c291f4d92a1e3d1b6a76e1a3f, State: Running, Role: LEADER
I20260806 03:55:33.693614 7838 consensus_queue.cc:237] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36939 } }
I20260806 03:55:33.693639 7828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [sys.catalog]: configured and running, proceeding with master startup.
Aug 06 03:55:33 dist-test-slave-bqcl krb5kdc[7701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988533, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:33.694067 7841 sys_catalog.cc:455] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 288e0f1c291f4d92a1e3d1b6a76e1a3f. Latest consensus state: current_term: 1 leader_uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36939 } } }
I20260806 03:55:33.694180 7841 sys_catalog.cc:458] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:33.694334 7842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "288e0f1c291f4d92a1e3d1b6a76e1a3f" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36939 } } }
I20260806 03:55:33.694389 7842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:33.695029 7848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:33.695477 7848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:33.698195 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:33.688209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58446 (local address 127.12.15.126:36939)
0806 03:55:33.688386 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:33.688395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:33.688419 (+ 24us) server_negotiation.cc:407] Connection header received
0806 03:55:33.688831 (+ 412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:33.688839 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:33.688897 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:33.689022 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:33.689690 (+ 668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:33.690658 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:33.691406 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:33.691811 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:33.691907 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:33.694897 (+ 2990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:33.694901 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:33.694902 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:33.694913 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:33.696584 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:33.696978 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:33.696981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:33.696981 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:33.697098 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:33.697304 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:33.697305 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:33.697305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:33.697543 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:33.697666 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:33.697809 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 03:55:33.697939 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":80,"threads_started":1}
I20260806 03:55:33.698251 7848 catalog_manager.cc:1347] Generated new cluster ID: 64e15e5bf74542a89f6214d8ad2be180
I20260806 03:55:33.698283 7848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:33.715342 7848 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:33.716099 7848 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:33.726737 7848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f: Generated new TSK 0
I20260806 03:55:33.726917 7848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:55:33.757627 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36939
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:33.838845 7870 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:33.839052 7870 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:33.839074 7870 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:33.840173 7870 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:33.840214 7870 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:33.840240 7870 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:55:33.841874 7870 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7870
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:33.842106 7870 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:33.842372 7870 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:33.847098 7877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:33.847108 7882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:33.847306 7878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:33.847229 7870 server_base.cc:1034] running on GCE node
I20260806 03:55:33.847534 7870 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:33.847779 7870 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:33.848932 7870 hybrid_clock.cc:648] HybridClock initialized: now 1785988533848918 us; error 30 us; skew 500 ppm
Aug 06 03:55:33 dist-test-slave-bqcl krb5kdc[7701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988533, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:33.851433 7870 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:55:33.851830 7870 webserver.cc:460] Webserver started at http://127.12.15.65:39209/ using document root <none> and password file <none>
I20260806 03:55:33.852020 7870 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:33.852066 7870 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:33.852205 7870 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:33.853236 7870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "bc87fb4fb30648df9a49f0b6044c1eb2"
format_stamp: "Formatted at 2026-08-06 03:55:33 on dist-test-slave-bqcl"
server_key: "7f72dceb96a53f8344a23ac8a42fb4e3"
server_key_iv: "20bf56e19fa9aaf79e1fea46c8976abe"
server_key_version: "encryptionkey@0"
I20260806 03:55:33.853521 7870 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "bc87fb4fb30648df9a49f0b6044c1eb2"
format_stamp: "Formatted at 2026-08-06 03:55:33 on dist-test-slave-bqcl"
server_key: "7f72dceb96a53f8344a23ac8a42fb4e3"
server_key_iv: "20bf56e19fa9aaf79e1fea46c8976abe"
server_key_version: "encryptionkey@0"
I20260806 03:55:33.854748 7870 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:33.855405 7891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:33.855556 7870 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:33.855633 7870 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "bc87fb4fb30648df9a49f0b6044c1eb2"
format_stamp: "Formatted at 2026-08-06 03:55:33 on dist-test-slave-bqcl"
server_key: "7f72dceb96a53f8344a23ac8a42fb4e3"
server_key_iv: "20bf56e19fa9aaf79e1fea46c8976abe"
server_key_version: "encryptionkey@0"
I20260806 03:55:33.855698 7870 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:33.870050 7870 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:33.872098 7870 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:33.872220 7870 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:33.872421 7870 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:33.872717 7870 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:33.872749 7870 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:33.872771 7870 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:33.872794 7870 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:33.878536 7870 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:36059
I20260806 03:55:33.878603 8064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:36059 every 8 connection(s)
I20260806 03:55:33.878873 7870 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:55:33.882714 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 7870
I20260806 03:55:33.882853 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:55:33.883044 12349 external_mini_cluster.cc:1428] Setting key 5558f6c1bc8f15a96e8810e28e059ec9
Aug 06 03:55:33 dist-test-slave-bqcl krb5kdc[7701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988533, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:33.887866 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:33.879481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:47371 (local address 127.12.15.126:36939)
0806 03:55:33.879523 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:33.879526 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:33.879762 (+ 236us) server_negotiation.cc:407] Connection header received
0806 03:55:33.880257 (+ 495us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:33.880258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:33.880272 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:33.880325 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:33.881218 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:33.881723 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:33.882744 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:33.883003 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:33.883058 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:33.885272 (+ 2214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:33.885274 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:33.885274 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:33.885281 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:33.886601 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:33.887001 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:33.887003 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:33.887003 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:33.887069 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:33.887284 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:33.887285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:33.887285 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:33.887413 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:33.887478 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:33.887693 (+ 215us) server_negotiation.cc:299] Negotiation successful
0806 03:55:33.887778 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:55:33.887953 8068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:33.879454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36939 (local address 127.12.15.65:47371)
0806 03:55:33.879645 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:33.879680 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:33.880164 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:33.880394 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:33.880398 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:33.880939 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:33.881122 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:33.881127 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:33.881806 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:33.881807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:33.882662 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:33.882665 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:33.883087 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:33.883088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:33.883208 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:33.883665 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:33.883673 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:33.885177 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:33.886689 (+ 1512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:33.886696 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:33.886698 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:33.886910 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:33.887133 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:33.887133 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:33.887134 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:33.887231 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:33.887470 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:33.887472 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:33.887582 (+ 110us) client_negotiation.cc:769] Sending connection context
0806 03:55:33.887668 (+ 86us) client_negotiation.cc:240] Negotiation successful
0806 03:55:33.887783 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":80,"threads_started":1}
I20260806 03:55:33.888453 8065 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36939
I20260806 03:55:33.888548 8065 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:33.888716 8065 heartbeater.cc:511] Master 127.12.15.126:36939 requested a full tablet report, sending...
I20260806 03:55:33.889252 7769 ts_manager.cc:194] Registered new tserver with Master: bc87fb4fb30648df9a49f0b6044c1eb2 (127.12.15.65:36059)
I20260806 03:55:33.890142 7769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:47371
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:55:33.909291 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36939
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:33.991432 8074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:33.991647 8074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:33.991670 8074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:33.992739 8074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:33.992772 8074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:33.992796 8074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:55:33.994275 8074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:33.994508 8074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:33.994696 8074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:33.999430 8086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:33.999532 8084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:33.999686 8074 server_base.cc:1034] running on GCE node
W20260806 03:55:33.999543 8088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:33.999930 8074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:34.000161 8074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:34.001320 8074 hybrid_clock.cc:648] HybridClock initialized: now 1785988534001309 us; error 27 us; skew 500 ppm
Aug 06 03:55:34 dist-test-slave-bqcl krb5kdc[7701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988534, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:34.003765 8074 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:55:34.004271 8074 webserver.cc:460] Webserver started at http://127.12.15.66:40959/ using document root <none> and password file <none>
I20260806 03:55:34.004454 8074 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:34.004493 8074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:34.004588 8074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:34.005589 8074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "0fb99bd2f3d844c096d1f298029a6a54"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "7edec714020291d3d9189b9cb61c25a7"
server_key_iv: "309cedf0c2f5d845a5bd59bb8806707c"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.005882 8074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "0fb99bd2f3d844c096d1f298029a6a54"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "7edec714020291d3d9189b9cb61c25a7"
server_key_iv: "309cedf0c2f5d845a5bd59bb8806707c"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.007141 8074 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:55:34.007803 8103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.007956 8074 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:34.008039 8074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "0fb99bd2f3d844c096d1f298029a6a54"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "7edec714020291d3d9189b9cb61c25a7"
server_key_iv: "309cedf0c2f5d845a5bd59bb8806707c"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.008104 8074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:34.025036 8074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:34.026804 8074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:34.026921 8074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:34.027132 8074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:34.027401 8074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:34.027433 8074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.027454 8074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:34.027477 8074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.032816 8074 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:34765
I20260806 03:55:34.032863 8275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:34765 every 8 connection(s)
I20260806 03:55:34.033154 8074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:55:34.034363 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 8074
I20260806 03:55:34.034471 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:55:34.034683 12349 external_mini_cluster.cc:1428] Setting key 54f4ed3e2828bbf9f332b1b69c360f8d
Aug 06 03:55:34 dist-test-slave-bqcl krb5kdc[7701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988534, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:34.042728 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.033797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:42303 (local address 127.12.15.126:36939)
0806 03:55:34.033854 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.033856 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.034040 (+ 184us) server_negotiation.cc:407] Connection header received
0806 03:55:34.034624 (+ 584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.034626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.034642 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.034724 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:34.035827 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.036478 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.037800 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.037989 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.038041 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.040173 (+ 2132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:34.040174 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:34.040175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:34.040182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:34.041435 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:34.041804 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:34.041805 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:34.041805 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:34.041858 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:34.042071 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:34.042072 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:34.042073 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:34.042214 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:34.042272 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.042530 (+ 258us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.042625 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260806 03:55:34.042858 8278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.033766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36939 (local address 127.12.15.66:42303)
0806 03:55:34.033919 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.033960 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.034455 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.034799 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.034803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.035553 (+ 750us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:34.035741 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.035746 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.036569 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.036570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.037720 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.037723 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.038068 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.038069 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.038229 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.038671 (+ 442us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:34.038681 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:34.040086 (+ 1405us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:34.041510 (+ 1424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:34.041517 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:34.041519 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:34.041738 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:34.041912 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:34.041912 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:34.041913 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:34.042016 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:34.042270 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:34.042277 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:34.042410 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.042499 (+ 89us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.042649 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":70,"threads_started":1}
I20260806 03:55:34.043293 8276 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36939
I20260806 03:55:34.043382 8276 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:34.043551 8276 heartbeater.cc:511] Master 127.12.15.126:36939 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
I20260806 03:55:34.043900 7769 ts_manager.cc:194] Registered new tserver with Master: 0fb99bd2f3d844c096d1f298029a6a54 (127.12.15.66:34765)
I20260806 03:55:34.044237 7769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:42303
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:55:34.060127 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36939
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:39687
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:34.139649 8288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:34.139848 8288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:34.139870 8288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:34.140971 8288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:34.141011 8288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:34.141036 8288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:55:34.142480 8288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:39687
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:34.142701 8288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:34.142906 8288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:34.147766 8296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:34.147837 8297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:34.147794 8288 server_base.cc:1034] running on GCE node
W20260806 03:55:34.147765 8299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:34.148168 8288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:34.148397 8288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:34.149564 8288 hybrid_clock.cc:648] HybridClock initialized: now 1785988534149550 us; error 24 us; skew 500 ppm
Aug 06 03:55:34 dist-test-slave-bqcl krb5kdc[7701](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988534, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:34.151855 8288 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:55:34.152263 8288 webserver.cc:460] Webserver started at http://127.12.15.67:34415/ using document root <none> and password file <none>
I20260806 03:55:34.152457 8288 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:34.152514 8288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:34.152637 8288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:34.153607 8288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "8c979307912c4e3e8d84fe758d766205"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "79022cd3e83e06da1bd786fddd0337d6"
server_key_iv: "156709d8e22e18364b6be4668defd537"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.153923 8288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "8c979307912c4e3e8d84fe758d766205"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "79022cd3e83e06da1bd786fddd0337d6"
server_key_iv: "156709d8e22e18364b6be4668defd537"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.155151 8288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:55:34.155830 8308 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.155972 8288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:34.156054 8288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "8c979307912c4e3e8d84fe758d766205"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "79022cd3e83e06da1bd786fddd0337d6"
server_key_iv: "156709d8e22e18364b6be4668defd537"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.156118 8288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:34.162352 8288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:34.164002 8288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:34.164130 8288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:34.164311 8288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:34.164575 8288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:34.164608 8288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.164634 8288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:34.164659 8288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.170068 8288 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:38511
I20260806 03:55:34.170111 8476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:38511 every 8 connection(s)
I20260806 03:55:34.170403 8288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:55:34.174635 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 8288
I20260806 03:55:34.174757 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:55:34.174952 12349 external_mini_cluster.cc:1428] Setting key 532806f9c2142cf031fdacd7f7291dfc
Aug 06 03:55:34 dist-test-slave-bqcl krb5kdc[7701](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988534, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:34.178740 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.171025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:51993 (local address 127.12.15.126:36939)
0806 03:55:34.171080 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.171082 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.171251 (+ 169us) server_negotiation.cc:407] Connection header received
0806 03:55:34.171699 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.171701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.171713 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.171760 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:34.172630 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.173119 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.174052 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.174315 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.174367 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.176642 (+ 2275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:34.176643 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:34.176644 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:34.176651 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:34.177671 (+ 1020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:34.177992 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:34.177993 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:34.177993 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:34.178037 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:34.178262 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:34.178263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:34.178263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:34.178366 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:34.178416 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.178620 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.178679 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:55:34.178869 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.170979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36939 (local address 127.12.15.67:51993)
0806 03:55:34.171147 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.171180 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.171605 (+ 425us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.171835 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.171838 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.172368 (+ 530us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:34.172539 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.172543 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.173199 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.173200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.173972 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.173974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.174393 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.174395 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.174561 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.175138 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:34.175146 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:34.176557 (+ 1411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:34.177739 (+ 1182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:34.177745 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:34.177747 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:34.177941 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:34.178078 (+ 137us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:34.178078 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:34.178078 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:34.178207 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:34.178411 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:34.178414 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:34.178516 (+ 102us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.178588 (+ 72us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.178696 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":80,"threads_started":1}
I20260806 03:55:34.179279 8478 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36939
I20260806 03:55:34.179360 8478 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:34.179490 8478 heartbeater.cc:511] Master 127.12.15.126:36939 requested a full tablet report, sending...
I20260806 03:55:34.179797 7769 ts_manager.cc:194] Registered new tserver with Master: 8c979307912c4e3e8d84fe758d766205 (127.12.15.67:38511)
I20260806 03:55:34.180154 7769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:51993
I20260806 03:55:34.186398 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:55:34.193071 7835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.187645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58450 (local address 127.12.15.126:36939)
0806 03:55:34.187702 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.187704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.187773 (+ 69us) server_negotiation.cc:407] Connection header received
0806 03:55:34.187826 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.187827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.187835 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.187876 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:34.188406 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.188911 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.189619 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.189801 (+ 182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.189855 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.191007 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:34.191008 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:34.191009 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:34.191015 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:34.192152 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:34.192476 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:34.192477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:34.192477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:34.192540 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:34.192678 (+ 138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:34.192679 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:34.192679 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:34.192788 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:34.192862 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.192942 (+ 80us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.192985 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260806 03:55:34.194622 7769 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:55:34.196210 7769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:55:34.202316 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.198501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54272 (local address 127.12.15.67:38511)
0806 03:55:34.198976 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.198980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.198990 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:55:34.199024 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.199026 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.199039 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.199126 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.199279 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.200017 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.201194 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.201911 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.201981 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.202042 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.202203 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.202243 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":83,"threads_started":1}
I20260806 03:55:34.202353 8487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.198141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:34765 (local address 127.0.0.1:53352)
0806 03:55:34.198501 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.198511 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.198561 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.198862 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.198865 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.198866 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.198938 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.198940 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.200215 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.200216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.201142 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.201145 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.202009 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.202010 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.202092 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.202099 (+ 7us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.202170 (+ 71us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.202233 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":5,"thread_start_us":51,"threads_started":1}
I20260806 03:55:34.202358 8489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.198141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38511 (local address 127.0.0.1:54272)
0806 03:55:34.198449 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.198490 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.198559 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.199178 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.199179 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.199181 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.199222 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.199223 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.200112 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.200113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.201124 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.201127 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.202000 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.202001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.202084 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.202097 (+ 13us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.202189 (+ 92us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.202236 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":281,"mutex_wait_us":3,"thread_start_us":51,"threads_started":1}
I20260806 03:55:34.202817 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.198327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53352 (local address 127.12.15.66:34765)
0806 03:55:34.198702 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.198707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.198717 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:55:34.198757 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.198760 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.198775 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.198846 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.199022 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.200096 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.201230 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.201910 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.202637 (+ 727us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.202703 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.202727 (+ 24us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.202760 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":90,"threads_started":1}
I20260806 03:55:34.202924 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.198335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36059 (local address 127.0.0.1:44728)
0806 03:55:34.198674 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.198682 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.198717 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.199412 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.199413 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.199415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.199447 (+ 32us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.199449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.200648 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.200649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.201601 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.201604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.202702 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.202703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.202770 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.202774 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.202819 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.202855 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":45,"threads_started":1}
I20260806 03:55:34.203245 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.198441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44728 (local address 127.12.15.65:36059)
0806 03:55:34.199266 (+ 825us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.199270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.199279 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:55:34.199315 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.199317 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.199331 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.199407 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.199509 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.200543 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.201683 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.202331 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.202997 (+ 666us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.203089 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.203118 (+ 29us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.203165 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":782,"thread_start_us":95,"threads_started":1}
I20260806 03:55:34.203353 8183 tablet_service.cc:1467] Processing CreateTablet for tablet 9ac011877d3841c5addf205c25e5d18b (DEFAULT_TABLE table=test-table [id=8ea93ce065744eeabe10a3723b33f8f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:34.203351 8381 tablet_service.cc:1467] Processing CreateTablet for tablet 9ac011877d3841c5addf205c25e5d18b (DEFAULT_TABLE table=test-table [id=8ea93ce065744eeabe10a3723b33f8f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:34.203708 8183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ac011877d3841c5addf205c25e5d18b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:34.203708 8381 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ac011877d3841c5addf205c25e5d18b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:34.207048 8497 tablet_bootstrap.cc:492] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: Bootstrap starting.
I20260806 03:55:34.207687 8497 tablet_bootstrap.cc:654] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:34.207832 7967 tablet_service.cc:1467] Processing CreateTablet for tablet 9ac011877d3841c5addf205c25e5d18b (DEFAULT_TABLE table=test-table [id=8ea93ce065744eeabe10a3723b33f8f5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:34.208138 7967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9ac011877d3841c5addf205c25e5d18b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:34.208171 8497 log.cc:824] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:34.208473 8496 tablet_bootstrap.cc:492] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Bootstrap starting.
I20260806 03:55:34.208972 8497 tablet_bootstrap.cc:492] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: No bootstrap required, opened a new log
I20260806 03:55:34.209056 8497 ts_tablet_manager.cc:1397] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:34.209091 8496 tablet_bootstrap.cc:654] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:34.209640 8496 log.cc:824] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:34.210474 8496 tablet_bootstrap.cc:492] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: No bootstrap required, opened a new log
I20260806 03:55:34.210543 8496 ts_tablet_manager.cc:1397] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:34.211031 8501 tablet_bootstrap.cc:492] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2: Bootstrap starting.
I20260806 03:55:34.211038 8497 raft_consensus.cc:348] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.211225 8497 raft_consensus.cc:374] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:34.211261 8497 raft_consensus.cc:729] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c979307912c4e3e8d84fe758d766205, State: Initialized, Role: FOLLOWER
I20260806 03:55:34.211387 8497 consensus_queue.cc:260] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.211620 8501 tablet_bootstrap.cc:654] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:34.211647 8497 ts_tablet_manager.cc:1428] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:55:34.211771 8478 heartbeater.cc:503] Master 127.12.15.126:36939 was elected leader, sending a full tablet report...
I20260806 03:55:34.212169 8501 log.cc:824] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:34.212283 8496 raft_consensus.cc:348] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.212409 8496 raft_consensus.cc:374] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:34.212443 8496 raft_consensus.cc:729] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fb99bd2f3d844c096d1f298029a6a54, State: Initialized, Role: FOLLOWER
I20260806 03:55:34.212595 8496 consensus_queue.cc:260] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.212874 8496 ts_tablet_manager.cc:1428] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260806 03:55:34.212893 8501 tablet_bootstrap.cc:492] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2: No bootstrap required, opened a new log
I20260806 03:55:34.212898 8276 heartbeater.cc:503] Master 127.12.15.126:36939 was elected leader, sending a full tablet report...
I20260806 03:55:34.212955 8501 ts_tablet_manager.cc:1397] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:34.214581 8501 raft_consensus.cc:348] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.214712 8501 raft_consensus.cc:374] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:34.214732 8501 raft_consensus.cc:729] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc87fb4fb30648df9a49f0b6044c1eb2, State: Initialized, Role: FOLLOWER
I20260806 03:55:34.214848 8501 consensus_queue.cc:260] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.215091 8501 ts_tablet_manager.cc:1428] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:55:34.215122 8065 heartbeater.cc:503] Master 127.12.15.126:36939 was elected leader, sending a full tablet report...
W20260806 03:55:34.283723 8277 tablet.cc:2369] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:55:34.299860 8502 raft_consensus.cc:482] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:34.299983 8502 raft_consensus.cc:504] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.300356 8502 leader_election.cc:288] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc87fb4fb30648df9a49f0b6044c1eb2 (127.12.15.65:36059), 0fb99bd2f3d844c096d1f298029a6a54 (127.12.15.66:34765)
I20260806 03:55:34.303190 8493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.300635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:52885 (local address 127.12.15.65:36059)
0806 03:55:34.300674 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.300676 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.300685 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:55:34.300712 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.300713 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.300724 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.300795 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.300961 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.301498 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.302463 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.302857 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.302914 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.302923 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.303084 (+ 161us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.303122 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:55:34.303192 8490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.300610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:47825 (local address 127.12.15.66:34765)
0806 03:55:34.300657 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.300660 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.300868 (+ 208us) server_negotiation.cc:407] Connection header received
0806 03:55:34.300902 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.300903 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.300916 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.300970 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.301115 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.301597 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.302502 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.302872 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.302938 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.302946 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.303089 (+ 143us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.303126 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260806 03:55:34.303205 8480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.300527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36059 (local address 127.12.15.67:52885)
0806 03:55:34.300596 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.300608 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.300651 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.300813 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.300814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.300816 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.300890 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.300892 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.301582 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.301583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.302375 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.302378 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.302933 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.302934 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.303011 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.303019 (+ 8us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.303066 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.303109 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":25,"mutex_wait_us":3}
I20260806 03:55:34.303212 8508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.300527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:34765 (local address 127.12.15.67:47825)
0806 03:55:34.300782 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.300793 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.300854 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.300979 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.300980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.300981 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.301052 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.301054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.301671 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.301672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.302413 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.302415 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.302950 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.302951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.303011 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.303017 (+ 6us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.303064 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.303105 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":51,"thread_start_us":131,"threads_started":1}
I20260806 03:55:34.303551 8210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ac011877d3841c5addf205c25e5d18b" candidate_uuid: "8c979307912c4e3e8d84fe758d766205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb99bd2f3d844c096d1f298029a6a54" is_pre_election: true
I20260806 03:55:34.303701 8210 raft_consensus.cc:2454] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c979307912c4e3e8d84fe758d766205 in term 0.
I20260806 03:55:34.303756 7999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ac011877d3841c5addf205c25e5d18b" candidate_uuid: "8c979307912c4e3e8d84fe758d766205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" is_pre_election: true
I20260806 03:55:34.303915 7999 raft_consensus.cc:2454] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c979307912c4e3e8d84fe758d766205 in term 0.
I20260806 03:55:34.303912 8313 leader_election.cc:302] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb99bd2f3d844c096d1f298029a6a54, 8c979307912c4e3e8d84fe758d766205; no voters:
I20260806 03:55:34.304045 8502 raft_consensus.cc:2781] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:55:34.304105 8502 raft_consensus.cc:482] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:55:34.304121 8502 raft_consensus.cc:3037] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:34.304854 8502 raft_consensus.cc:504] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.304962 8502 leader_election.cc:288] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [CANDIDATE]: Term 1 election: Requested vote from peers bc87fb4fb30648df9a49f0b6044c1eb2 (127.12.15.65:36059), 0fb99bd2f3d844c096d1f298029a6a54 (127.12.15.66:34765)
I20260806 03:55:34.305155 8210 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ac011877d3841c5addf205c25e5d18b" candidate_uuid: "8c979307912c4e3e8d84fe758d766205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0fb99bd2f3d844c096d1f298029a6a54"
I20260806 03:55:34.305239 8210 raft_consensus.cc:3037] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:34.305928 8210 raft_consensus.cc:2454] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c979307912c4e3e8d84fe758d766205 in term 1.
I20260806 03:55:34.306032 7999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9ac011877d3841c5addf205c25e5d18b" candidate_uuid: "8c979307912c4e3e8d84fe758d766205" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2"
I20260806 03:55:34.306110 7999 raft_consensus.cc:3037] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:34.306113 8313 leader_election.cc:302] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0fb99bd2f3d844c096d1f298029a6a54, 8c979307912c4e3e8d84fe758d766205; no voters:
I20260806 03:55:34.306252 8502 raft_consensus.cc:2781] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:34.306439 8502 raft_consensus.cc:686] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 1 LEADER]: Becoming Leader. State: Replica: 8c979307912c4e3e8d84fe758d766205, State: Running, Role: LEADER
I20260806 03:55:34.306547 8502 consensus_queue.cc:237] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } }
I20260806 03:55:34.306955 7999 raft_consensus.cc:2454] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c979307912c4e3e8d84fe758d766205 in term 1.
I20260806 03:55:34.307624 7769 catalog_manager.cc:5495] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c979307912c4e3e8d84fe758d766205 (127.12.15.67). New cstate: current_term: 1 leader_uuid: "8c979307912c4e3e8d84fe758d766205" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c979307912c4e3e8d84fe758d766205" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38511 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 } health_report { overall_health: UNKNOWN } } }
I20260806 03:55:34.320108 8495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.318160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54278 (local address 127.12.15.67:38511)
0806 03:55:34.318211 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.318213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.318220 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:55:34.318247 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.318247 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.318263 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.318320 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:55:34.318413 (+ 93us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.318858 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.319539 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.319710 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.319751 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.319843 (+ 92us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:55:34.319923 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:55:34.319987 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.320008 (+ 21us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.320016 (+ 8us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260806 03:55:34.321744 8210 raft_consensus.cc:1261] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 1 FOLLOWER]: Refusing update from remote peer 8c979307912c4e3e8d84fe758d766205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:34.321746 7999 raft_consensus.cc:1261] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [term 1 FOLLOWER]: Refusing update from remote peer 8c979307912c4e3e8d84fe758d766205: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:34.322036 8502 consensus_queue.cc:1035] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc87fb4fb30648df9a49f0b6044c1eb2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 36059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:34.322203 8509 consensus_queue.cc:1035] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0fb99bd2f3d844c096d1f298029a6a54" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 34765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:34.324724 8517 mvcc.cc:205] Tried to move back new op lower bound from 7315409036579512320 to 7315409036520394752. Current Snapshot: MvccSnapshot[applied={T|T < 7315409036579512320}]
I20260806 03:55:34.324865 8514 mvcc.cc:205] Tried to move back new op lower bound from 7315409036579512320 to 7315409036520394752. Current Snapshot: MvccSnapshot[applied={T|T < 7315409036579512320 or (T in {7315409036579512320})}]
I20260806 03:55:34.325039 8516 mvcc.cc:205] Tried to move back new op lower bound from 7315409036579512320 to 7315409036520394752. Current Snapshot: MvccSnapshot[applied={T|T < 7315409036579512320}]
I20260806 03:55:34.326306 7769 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:34.326391 7769 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:34.327057 7769 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 288e0f1c291f4d92a1e3d1b6a76e1a3f: Sending DeleteTablet for 3 replicas of tablet 9ac011877d3841c5addf205c25e5d18b
I20260806 03:55:34.327384 8381 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ac011877d3841c5addf205c25e5d18b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:34 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:54272
I20260806 03:55:34.327404 7967 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ac011877d3841c5addf205c25e5d18b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:34 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:44728
I20260806 03:55:34.327615 8183 tablet_service.cc:1514] Processing DeleteTablet for tablet 9ac011877d3841c5addf205c25e5d18b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:34 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:53352
I20260806 03:55:34.327653 8535 tablet_replica.cc:330] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: stopping tablet replica
I20260806 03:55:34.327798 8535 raft_consensus.cc:2229] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:55:34.327843 8534 tablet_replica.cc:330] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2: stopping tablet replica
I20260806 03:55:34.327944 8534 raft_consensus.cc:2229] T 9ac011877d3841c5addf205c25e5d18b P bc87fb4fb30648df9a49f0b6044c1eb2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:34.327953 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 7870
I20260806 03:55:34.328050 8535 raft_consensus.cc:2258] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:34.330772 8536 tablet_replica.cc:330] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: stopping tablet replica
I20260806 03:55:34.330874 8536 raft_consensus.cc:2229] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:34.330788 8535 ts_tablet_manager.cc:1905] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:34.330971 8536 raft_consensus.cc:2258] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:34.331419 8536 ts_tablet_manager.cc:1905] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:34.332875 8535 ts_tablet_manager.cc:1918] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:34.333001 8535 log.cc:1196] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/9ac011877d3841c5addf205c25e5d18b
I20260806 03:55:34.333112 8536 ts_tablet_manager.cc:1918] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:34.333201 8536 log.cc:1196] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/9ac011877d3841c5addf205c25e5d18b
I20260806 03:55:34.333472 8535 ts_tablet_manager.cc:1939] T 9ac011877d3841c5addf205c25e5d18b P 8c979307912c4e3e8d84fe758d766205: Deleting consensus metadata
I20260806 03:55:34.333521 8536 ts_tablet_manager.cc:1939] T 9ac011877d3841c5addf205c25e5d18b P 0fb99bd2f3d844c096d1f298029a6a54: Deleting consensus metadata
I20260806 03:55:34.333956 7746 catalog_manager.cc:4841] TS 8c979307912c4e3e8d84fe758d766205 (127.12.15.67:38511): tablet 9ac011877d3841c5addf205c25e5d18b (table test-table [id=8ea93ce065744eeabe10a3723b33f8f5]) successfully deleted
I20260806 03:55:34.333989 7749 catalog_manager.cc:4841] TS 0fb99bd2f3d844c096d1f298029a6a54 (127.12.15.66:34765): tablet 9ac011877d3841c5addf205c25e5d18b (table test-table [id=8ea93ce065744eeabe10a3723b33f8f5]) successfully deleted
W20260806 03:55:34.343158 7746 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:36059 (error 108)
I20260806 03:55:34.343598 8488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.343485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:36059 (local address 127.0.0.1:44736)
0806 03:55:34.343532 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.343567 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:36059: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 03:55:34.343731 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 8074
W20260806 03:55:34.343781 7746 catalog_manager.cc:4584] TS bc87fb4fb30648df9a49f0b6044c1eb2 (127.12.15.65:36059): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9ac011877d3841c5addf205c25e5d18b: Network error: Client connection negotiation failed: client connection to 127.12.15.65:36059: connect: Connection refused (error 111)
I20260806 03:55:34.350085 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 8288
I20260806 03:55:34.360947 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 7719
2026-08-06T03:55:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2047 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-06T03:55:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:34Z Disabled control of system clock
I20260806 03:55:34.392422 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36859
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:40445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:36859
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:34.471969 8548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:34.472179 8548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:34.472198 8548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:34.473268 8548 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:34.473302 8548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:34.473313 8548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:34.473325 8548 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:34.473340 8548 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:34.474619 8548 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:36859
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:34.474807 8548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:34.475030 8548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:34.479708 8556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:34.479709 8557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:34.479820 8548 server_base.cc:1034] running on GCE node
W20260806 03:55:34.479779 8559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:34.480190 8548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:34.480453 8548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:34.481612 8548 hybrid_clock.cc:648] HybridClock initialized: now 1785988534481597 us; error 30 us; skew 500 ppm
I20260806 03:55:34.485486 8548 webserver.cc:460] Webserver started at http://127.12.15.126:42787/ using document root <none> and password file <none>
I20260806 03:55:34.485715 8548 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:34.485766 8548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:34.485893 8548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:34.486976 8548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "f9aa77b266514e4ba63c06512aef35e5"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "fcc93116f1aafa3542d2500a70dea356"
server_key_iv: "410e7a7ded815236152f8369e318a2bf"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.487316 8548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "f9aa77b266514e4ba63c06512aef35e5"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "fcc93116f1aafa3542d2500a70dea356"
server_key_iv: "410e7a7ded815236152f8369e318a2bf"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.488535 8548 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:55:34.489189 8570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.489341 8548 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:34.489428 8548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "f9aa77b266514e4ba63c06512aef35e5"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "fcc93116f1aafa3542d2500a70dea356"
server_key_iv: "410e7a7ded815236152f8369e318a2bf"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.489494 8548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:34.501370 8548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:34.501636 8548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:34.501760 8548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:34.505504 8548 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:36859
I20260806 03:55:34.505535 8659 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:36859 every 8 connection(s)
I20260806 03:55:34.505836 8548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:34.506420 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 8548
I20260806 03:55:34.506549 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:34.506731 12349 external_mini_cluster.cc:1428] Setting key d6e31b3cdb80d01f68f87a205af4897c
I20260806 03:55:34.506860 8663 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:34.509879 8663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5: Bootstrap starting.
I20260806 03:55:34.510545 8663 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:34.510850 8663 log.cc:824] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:34.511600 8663 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5: No bootstrap required, opened a new log
I20260806 03:55:34.512033 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.507226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59098 (local address 127.12.15.126:36859)
0806 03:55:34.507402 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.507411 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.507438 (+ 27us) server_negotiation.cc:407] Connection header received
0806 03:55:34.507932 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.507939 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.507997 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.508111 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:34.508331 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.509169 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.510254 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.510671 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.510776 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.511457 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:34.511461 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:34.511461 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:34.511608 (+ 147us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:34.511639 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:34.511649 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:34.511704 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.511788 (+ 84us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.511865 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260806 03:55:34.513602 8663 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9aa77b266514e4ba63c06512aef35e5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36859 } }
I20260806 03:55:34.513737 8663 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:34.513770 8663 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9aa77b266514e4ba63c06512aef35e5, State: Initialized, Role: FOLLOWER
I20260806 03:55:34.513885 8663 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9aa77b266514e4ba63c06512aef35e5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36859 } }
I20260806 03:55:34.513944 8663 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:34.513973 8663 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:34.514019 8663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:34.514555 8663 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9aa77b266514e4ba63c06512aef35e5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36859 } }
I20260806 03:55:34.514654 8663 leader_election.cc:302] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9aa77b266514e4ba63c06512aef35e5; no voters:
I20260806 03:55:34.514816 8663 leader_election.cc:288] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:34.514863 8668 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:34.515017 8668 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [term 1 LEADER]: Becoming Leader. State: Replica: f9aa77b266514e4ba63c06512aef35e5, State: Running, Role: LEADER
I20260806 03:55:34.515064 8663 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:34.515137 8668 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9aa77b266514e4ba63c06512aef35e5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36859 } }
I20260806 03:55:34.515465 8669 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9aa77b266514e4ba63c06512aef35e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9aa77b266514e4ba63c06512aef35e5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36859 } } }
I20260806 03:55:34.515483 8670 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9aa77b266514e4ba63c06512aef35e5. Latest consensus state: current_term: 1 leader_uuid: "f9aa77b266514e4ba63c06512aef35e5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9aa77b266514e4ba63c06512aef35e5" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36859 } } }
I20260806 03:55:34.515648 8670 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:34.515599 8669 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:34.516131 8677 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:34.516595 8677 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:34.517961 8677 catalog_manager.cc:1347] Generated new cluster ID: 7573060fe47b473aa6101d6ee09fee01
I20260806 03:55:34.518013 8677 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:34.535748 8677 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:34.536448 8677 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:34.543442 8677 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5: Generated new TSK 0
I20260806 03:55:34.543638 8677 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:34.548460 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36859
--builtin_ntp_servers=127.12.15.84:40445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:34.631451 8696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:34.631669 8696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:34.631692 8696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:34.632810 8696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:34.632855 8696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:34.632881 8696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:55:34.634292 8696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:34.634541 8696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:34.634852 8696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:34.640579 8703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:34.640584 8707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:34.640779 8696 server_base.cc:1034] running on GCE node
W20260806 03:55:34.640813 8705 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:34.641068 8696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:34.641317 8696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:34.642485 8696 hybrid_clock.cc:648] HybridClock initialized: now 1785988534642457 us; error 37 us; skew 500 ppm
I20260806 03:55:34.643695 8696 webserver.cc:460] Webserver started at http://127.12.15.65:42693/ using document root <none> and password file <none>
I20260806 03:55:34.643918 8696 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:34.643967 8696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:34.644094 8696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:34.645112 8696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "0007c42ba500424a98d588db2499291a"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "cb7b0483e61591cf7dbf4bfc0754b824"
server_key_iv: "f526a09319a58aa8d93284f05c2a1a3a"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.645442 8696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "0007c42ba500424a98d588db2499291a"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "cb7b0483e61591cf7dbf4bfc0754b824"
server_key_iv: "f526a09319a58aa8d93284f05c2a1a3a"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.646703 8696 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:34.647367 8716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.647540 8696 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:34.647624 8696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "0007c42ba500424a98d588db2499291a"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "cb7b0483e61591cf7dbf4bfc0754b824"
server_key_iv: "f526a09319a58aa8d93284f05c2a1a3a"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.647686 8696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:34.657550 8696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:34.657796 8696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:34.657920 8696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:34.658105 8696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:34.658428 8696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:34.658461 8696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.658488 8696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:34.658521 8696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.663973 8696 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:38165
I20260806 03:55:34.664008 8902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:38165 every 8 connection(s)
I20260806 03:55:34.664319 8696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:55:34.669368 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.664954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:48425 (local address 127.12.15.126:36859)
0806 03:55:34.665008 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.665011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.665203 (+ 192us) server_negotiation.cc:407] Connection header received
0806 03:55:34.665718 (+ 515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.665719 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.665732 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.665783 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:34.666488 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.667078 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.668030 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.668240 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.668331 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.669066 (+ 735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:34.669068 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:34.669068 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:34.669073 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:34.669091 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:34.669096 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:34.669150 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.669236 (+ 86us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.669280 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:55:34.669577 8907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.664927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36859 (local address 127.12.15.65:48425)
0806 03:55:34.665092 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.665124 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.665612 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.665851 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.665855 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.665865 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:34.666404 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.666410 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.667167 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.667168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.667937 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.667940 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.668310 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.668310 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.668445 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.668904 (+ 459us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:34.668904 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:34.668933 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:34.668956 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:34.669157 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:34.669162 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:34.669165 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.669240 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.669347 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":70,"threads_started":1}
I20260806 03:55:34.670092 8905 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36859
I20260806 03:55:34.670214 8905 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:34.670387 8905 heartbeater.cc:511] Master 127.12.15.126:36859 requested a full tablet report, sending...
I20260806 03:55:34.670893 8601 ts_manager.cc:194] Registered new tserver with Master: 0007c42ba500424a98d588db2499291a (127.12.15.65:38165)
I20260806 03:55:34.671507 8601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.65:48425
I20260806 03:55:34.674214 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 8696
I20260806 03:55:34.674319 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:55:34.674510 12349 external_mini_cluster.cc:1428] Setting key e1512ea9cc3fbbe5579561d62d7e920e
I20260806 03:55:34.675359 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36859
--builtin_ntp_servers=127.12.15.84:40445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:34.758627 8908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:34.758908 8908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:34.758940 8908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:34.760053 8908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:34.760104 8908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:34.760138 8908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:55:34.761471 8908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:34.761687 8908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:34.761956 8908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:34.766618 8915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:34.766592 8918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:34.766584 8916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:34.767020 8908 server_base.cc:1034] running on GCE node
I20260806 03:55:34.767211 8908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:34.767396 8908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:34.768533 8908 hybrid_clock.cc:648] HybridClock initialized: now 1785988534768517 us; error 28 us; skew 500 ppm
I20260806 03:55:34.769667 8908 webserver.cc:460] Webserver started at http://127.12.15.66:35269/ using document root <none> and password file <none>
I20260806 03:55:34.769867 8908 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:34.769910 8908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:34.770040 8908 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:34.771102 8908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "070608d492aea579758b07b37eb6abad"
server_key_iv: "412918c7a5fa2c1fc547c7ddb6a4bfe8"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.771421 8908 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "070608d492aea579758b07b37eb6abad"
server_key_iv: "412918c7a5fa2c1fc547c7ddb6a4bfe8"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.772569 8908 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:55:34.773211 8925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.773375 8908 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:34.773458 8908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "070608d492aea579758b07b37eb6abad"
server_key_iv: "412918c7a5fa2c1fc547c7ddb6a4bfe8"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.773509 8908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:34.782938 8908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:34.783164 8908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:34.783258 8908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:34.783416 8908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:34.783671 8908 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:34.783701 8908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.783726 8908 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:34.783743 8908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.789099 8908 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:36977
I20260806 03:55:34.789175 9116 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:36977 every 8 connection(s)
I20260806 03:55:34.789454 8908 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:55:34.790544 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 8908
I20260806 03:55:34.790632 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:55:34.790812 12349 external_mini_cluster.cc:1428] Setting key 2d2c22feb8848f535fa12d99549c8187
I20260806 03:55:34.791705 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36859
--builtin_ntp_servers=127.12.15.84:40445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 03:55:34.794909 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.790165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:60405 (local address 127.12.15.126:36859)
0806 03:55:34.790209 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.790211 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.790431 (+ 220us) server_negotiation.cc:407] Connection header received
0806 03:55:34.790970 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.790972 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.790984 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.791044 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:34.791699 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.792316 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.793482 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.793689 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.793745 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.794642 (+ 897us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:34.794644 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:34.794644 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:34.794648 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:34.794677 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:34.794681 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:34.794726 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.794799 (+ 73us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.794826 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:55:34.795122 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.790103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36859 (local address 127.12.15.66:60405)
0806 03:55:34.790299 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.790331 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.790871 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.791113 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.791117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.791128 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:34.791610 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.791614 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.792402 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.792403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.793404 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.793407 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.793767 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.793768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.793917 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.794508 (+ 591us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:34.794509 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:34.794539 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:34.794561 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:34.794733 (+ 172us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:34.794739 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:34.794742 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.794804 (+ 62us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.794907 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":102,"threads_started":1}
I20260806 03:55:34.795545 9117 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36859
I20260806 03:55:34.795631 9117 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:34.795807 9117 heartbeater.cc:511] Master 127.12.15.126:36859 requested a full tablet report, sending...
I20260806 03:55:34.796151 8601 ts_manager.cc:194] Registered new tserver with Master: bbb5dd0f1cb5444c8e82d1d3401874c4 (127.12.15.66:36977)
I20260806 03:55:34.796497 8601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.66:60405
W20260806 03:55:34.872669 9122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:34.872864 9122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:34.872893 9122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:34.873981 9122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:34.874018 9122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:34.874050 9122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:55:34.875360 9122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:40445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:36859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:34.875586 9122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:34.875883 9122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:34.880492 9130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:34.880514 9131 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:34.880555 9133 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:34.880606 9122 server_base.cc:1034] running on GCE node
I20260806 03:55:34.880905 9122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:34.881108 9122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:34.882261 9122 hybrid_clock.cc:648] HybridClock initialized: now 1785988534882244 us; error 29 us; skew 500 ppm
I20260806 03:55:34.883447 9122 webserver.cc:460] Webserver started at http://127.12.15.67:33879/ using document root <none> and password file <none>
I20260806 03:55:34.883649 9122 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:34.883708 9122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:34.883834 9122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:34.884831 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "e156670f24df485bbb593ea570a0484a"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "e02b77734c43996d25c8a13d25a4ad5c"
server_key_iv: "94b647d2f642762700d4e166fc96ba11"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.885146 9122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "e156670f24df485bbb593ea570a0484a"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "e02b77734c43996d25c8a13d25a4ad5c"
server_key_iv: "94b647d2f642762700d4e166fc96ba11"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.886435 9122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:55:34.887086 9140 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.887234 9122 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:34.887316 9122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "e156670f24df485bbb593ea570a0484a"
format_stamp: "Formatted at 2026-08-06 03:55:34 on dist-test-slave-bqcl"
server_key: "e02b77734c43996d25c8a13d25a4ad5c"
server_key_iv: "94b647d2f642762700d4e166fc96ba11"
server_key_version: "encryptionkey@0"
I20260806 03:55:34.887377 9122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:34.914752 9122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:34.915100 9122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:34.915243 9122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:34.915441 9122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:34.915771 9122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:34.915820 9122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.915848 9122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:34.915874 9122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:34.921116 9122 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:42421
I20260806 03:55:34.921188 9303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:42421 every 8 connection(s)
I20260806 03:55:34.921468 9122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:55:34.926811 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.922170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:34609 (local address 127.12.15.126:36859)
0806 03:55:34.922217 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.922219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.922444 (+ 225us) server_negotiation.cc:407] Connection header received
0806 03:55:34.922906 (+ 462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.922907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.922919 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.922969 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:34.923755 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.924393 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.925372 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.925584 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.925641 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.926468 (+ 827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:34.926470 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:34.926471 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:34.926520 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:34.926537 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:34.926542 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:34.926597 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.926691 (+ 94us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.926722 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:55:34.927032 9306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.922147 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36859 (local address 127.12.15.67:34609)
0806 03:55:34.922323 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.922359 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.922828 (+ 469us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.923035 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.923038 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.923047 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:34.923646 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.923653 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.924481 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.924482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.925284 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.925286 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.925658 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.925659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.925794 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.926312 (+ 518us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:34.926313 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:34.926343 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:34.926367 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:34.926605 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:34.926612 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:34.926615 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.926688 (+ 73us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.926811 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":73,"threads_started":1}
I20260806 03:55:34.927498 9304 heartbeater.cc:347] Connected to a master server at 127.12.15.126:36859
I20260806 03:55:34.927587 9304 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:34.927772 9304 heartbeater.cc:511] Master 127.12.15.126:36859 requested a full tablet report, sending...
I20260806 03:55:34.928117 8601 ts_manager.cc:194] Registered new tserver with Master: e156670f24df485bbb593ea570a0484a (127.12.15.67:42421)
I20260806 03:55:34.928367 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 9122
I20260806 03:55:34.928462 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:55:34.928534 8601 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.67:34609
I20260806 03:55:34.928687 12349 external_mini_cluster.cc:1428] Setting key ca015d596669b3470fe28b170f8e8776
I20260806 03:55:34.929644 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:55:34.933424 8666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.930935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59108 (local address 127.12.15.126:36859)
0806 03:55:34.931003 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.931005 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.931092 (+ 87us) server_negotiation.cc:407] Connection header received
0806 03:55:34.931142 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.931143 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.931151 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.931198 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:34.931334 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.931774 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.932475 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.932641 (+ 166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.932685 (+ 44us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.933209 (+ 524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:34.933210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:34.933211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:34.933215 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:34.933228 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:34.933232 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:34.933294 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.933343 (+ 49us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.933352 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 03:55:34.934806 8601 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:55:34.936312 8601 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:55:34.941432 9317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.938206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:38165 (local address 127.0.0.1:48412)
0806 03:55:34.938327 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.938365 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.938429 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.938620 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.938623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.938625 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.938703 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.938705 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.939679 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.939680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.940485 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.940488 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.941203 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.941204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.941283 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.941293 (+ 10us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.941328 (+ 35us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.941362 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":24,"thread_start_us":77,"threads_started":1}
I20260806 03:55:34.941864 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.938299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48412 (local address 127.12.15.65:38165)
0806 03:55:34.938481 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.938485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.938495 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:55:34.938532 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.938534 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.938547 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.938609 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.938770 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.939602 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.940569 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.941097 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.941617 (+ 520us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.941716 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.941743 (+ 27us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.941778 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":90,"threads_started":1}
I20260806 03:55:34.942158 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.938543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38550 (local address 127.12.15.67:42421)
0806 03:55:34.938791 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.938795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.938805 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:55:34.938838 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.938840 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.938854 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.938924 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.939041 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.939763 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.941093 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.941754 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.941789 (+ 35us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.941842 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.942030 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.942064 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":79,"threads_started":1}
I20260806 03:55:34.942353 9320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.938212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:42421 (local address 127.0.0.1:38550)
0806 03:55:34.938557 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.938568 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.938617 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.938925 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.938926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.938928 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.938968 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.938969 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.939837 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.939838 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.941004 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.941007 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.941890 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.941890 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.941946 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.941950 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.942244 (+ 294us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.942290 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":80,"threads_started":1}
I20260806 03:55:34.942535 8794 tablet_service.cc:1467] Processing CreateTablet for tablet d06751829cf6421eb84595cf4d2829c1 (DEFAULT_TABLE table=test-table [id=34f2f53e0c3b402e948db823680f215f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:34.942813 8794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d06751829cf6421eb84595cf4d2829c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:34.943200 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.938423 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:36977 (local address 127.0.0.1:35332)
0806 03:55:34.939874 (+ 1451us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.939884 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.939928 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.940182 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.940184 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.940186 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.940225 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.940227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.941404 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.941405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.942358 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.942361 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.942990 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.942991 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.943049 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.943053 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.943097 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.943133 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1431,"mutex_wait_us":2,"thread_start_us":64,"threads_started":1}
I20260806 03:55:34.943234 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.938756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.12.15.66:36977)
0806 03:55:34.938937 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.938941 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.939979 (+ 1038us) server_negotiation.cc:407] Connection header received
0806 03:55:34.940031 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.940033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.940048 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.940115 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.940292 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.941330 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.942440 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.942912 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.942975 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.943059 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.943110 (+ 51us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.943145 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":87,"threads_started":1}
I20260806 03:55:34.943907 9012 tablet_service.cc:1467] Processing CreateTablet for tablet d06751829cf6421eb84595cf4d2829c1 (DEFAULT_TABLE table=test-table [id=34f2f53e0c3b402e948db823680f215f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:34.944173 9012 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d06751829cf6421eb84595cf4d2829c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:34.945391 9326 tablet_bootstrap.cc:492] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a: Bootstrap starting.
I20260806 03:55:34.945796 9209 tablet_service.cc:1467] Processing CreateTablet for tablet d06751829cf6421eb84595cf4d2829c1 (DEFAULT_TABLE table=test-table [id=34f2f53e0c3b402e948db823680f215f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:34.946031 9209 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d06751829cf6421eb84595cf4d2829c1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:34.946041 9326 tablet_bootstrap.cc:654] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:34.946425 9326 log.cc:824] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:34.946907 9329 tablet_bootstrap.cc:492] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: Bootstrap starting.
I20260806 03:55:34.947412 9329 tablet_bootstrap.cc:654] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:34.947615 9329 log.cc:824] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:34.948197 9329 tablet_bootstrap.cc:492] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: No bootstrap required, opened a new log
I20260806 03:55:34.948241 9329 ts_tablet_manager.cc:1397] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260806 03:55:34.948411 9332 tablet_bootstrap.cc:492] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: Bootstrap starting.
I20260806 03:55:34.948987 9332 tablet_bootstrap.cc:654] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:34.949286 9332 log.cc:824] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:34.949541 9326 tablet_bootstrap.cc:492] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a: No bootstrap required, opened a new log
I20260806 03:55:34.949615 9326 ts_tablet_manager.cc:1397] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260806 03:55:34.949977 9332 tablet_bootstrap.cc:492] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: No bootstrap required, opened a new log
I20260806 03:55:34.950099 9332 ts_tablet_manager.cc:1397] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:34.950603 9329 raft_consensus.cc:348] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.950764 9329 raft_consensus.cc:374] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:34.950806 9329 raft_consensus.cc:729] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbb5dd0f1cb5444c8e82d1d3401874c4, State: Initialized, Role: FOLLOWER
I20260806 03:55:34.950937 9329 consensus_queue.cc:260] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.951207 9329 ts_tablet_manager.cc:1428] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260806 03:55:34.951227 9117 heartbeater.cc:503] Master 127.12.15.126:36859 was elected leader, sending a full tablet report...
I20260806 03:55:34.951962 9326 raft_consensus.cc:348] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.952098 9326 raft_consensus.cc:374] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:34.952131 9326 raft_consensus.cc:729] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0007c42ba500424a98d588db2499291a, State: Initialized, Role: FOLLOWER
I20260806 03:55:34.952235 9326 consensus_queue.cc:260] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.952494 8905 heartbeater.cc:503] Master 127.12.15.126:36859 was elected leader, sending a full tablet report...
I20260806 03:55:34.952426 9332 raft_consensus.cc:348] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.952541 9332 raft_consensus.cc:374] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:34.952579 9332 raft_consensus.cc:729] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e156670f24df485bbb593ea570a0484a, State: Initialized, Role: FOLLOWER
I20260806 03:55:34.952682 9332 consensus_queue.cc:260] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.952716 9326 ts_tablet_manager.cc:1428] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:55:34.952929 9332 ts_tablet_manager.cc:1428] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:55:34.952978 9304 heartbeater.cc:503] Master 127.12.15.126:36859 was elected leader, sending a full tablet report...
I20260806 03:55:34.974176 9335 raft_consensus.cc:482] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:34.974267 9335 raft_consensus.cc:504] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.974586 9335 leader_election.cc:288] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0007c42ba500424a98d588db2499291a (127.12.15.65:38165), e156670f24df485bbb593ea570a0484a (127.12.15.67:42421)
I20260806 03:55:34.977262 9321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.974822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:36765 (local address 127.12.15.67:42421)
0806 03:55:34.974877 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.974879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.974958 (+ 79us) server_negotiation.cc:407] Connection header received
0806 03:55:34.975016 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.975017 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.975026 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.975069 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.975194 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.975617 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.976604 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.976946 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.977005 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.977013 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.977139 (+ 126us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.977180 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:55:34.977257 9338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.974737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:42421 (local address 127.12.15.66:36765)
0806 03:55:34.974898 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.974907 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.974960 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.975076 (+ 116us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.975077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.975078 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.975140 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.975142 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.975686 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.975687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.976528 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.976531 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.977019 (+ 488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.977020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.977083 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.977086 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.977128 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.977166 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"mutex_wait_us":35,"thread_start_us":89,"threads_started":1}
I20260806 03:55:34.977381 9121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.974726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:38165 (local address 127.12.15.66:32849)
0806 03:55:34.974773 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:34.974785 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:34.974830 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:34.974969 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:34.974970 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:34.974972 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:34.975045 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.975046 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.975700 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.975701 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.976566 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:34.976570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.977139 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.977140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:34.977215 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.977218 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:34.977271 (+ 53us) client_negotiation.cc:240] Negotiation successful
0806 03:55:34.977312 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 03:55:34.977394 9319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.974814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:32849 (local address 127.12.15.65:38165)
0806 03:55:34.974864 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.974865 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.974874 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:55:34.974896 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.974897 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.974907 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.974961 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:34.975109 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.975618 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.976643 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.977067 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.977138 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.977145 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.977301 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.977339 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260806 03:55:34.977712 9239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d06751829cf6421eb84595cf4d2829c1" candidate_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e156670f24df485bbb593ea570a0484a" is_pre_election: true
I20260806 03:55:34.977749 8826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d06751829cf6421eb84595cf4d2829c1" candidate_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0007c42ba500424a98d588db2499291a" is_pre_election: true
I20260806 03:55:34.977850 8826 raft_consensus.cc:2454] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbb5dd0f1cb5444c8e82d1d3401874c4 in term 0.
I20260806 03:55:34.977837 9239 raft_consensus.cc:2454] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bbb5dd0f1cb5444c8e82d1d3401874c4 in term 0.
I20260806 03:55:34.978075 8930 leader_election.cc:302] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbb5dd0f1cb5444c8e82d1d3401874c4, e156670f24df485bbb593ea570a0484a; no voters:
I20260806 03:55:34.978232 9335 raft_consensus.cc:2781] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:55:34.978284 9335 raft_consensus.cc:482] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:55:34.978300 9335 raft_consensus.cc:3037] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:34.978922 9335 raft_consensus.cc:504] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.979100 9335 leader_election.cc:288] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [CANDIDATE]: Term 1 election: Requested vote from peers 0007c42ba500424a98d588db2499291a (127.12.15.65:38165), e156670f24df485bbb593ea570a0484a (127.12.15.67:42421)
I20260806 03:55:34.979192 8826 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d06751829cf6421eb84595cf4d2829c1" candidate_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0007c42ba500424a98d588db2499291a"
I20260806 03:55:34.979240 9239 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d06751829cf6421eb84595cf4d2829c1" candidate_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e156670f24df485bbb593ea570a0484a"
I20260806 03:55:34.979282 8826 raft_consensus.cc:3037] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:34.979307 9239 raft_consensus.cc:3037] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:34.979938 9239 raft_consensus.cc:2454] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbb5dd0f1cb5444c8e82d1d3401874c4 in term 1.
I20260806 03:55:34.980079 8826 raft_consensus.cc:2454] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bbb5dd0f1cb5444c8e82d1d3401874c4 in term 1.
I20260806 03:55:34.980134 8930 leader_election.cc:302] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bbb5dd0f1cb5444c8e82d1d3401874c4, e156670f24df485bbb593ea570a0484a; no voters:
I20260806 03:55:34.980225 9335 raft_consensus.cc:2781] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:34.980351 9335 raft_consensus.cc:686] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 1 LEADER]: Becoming Leader. State: Replica: bbb5dd0f1cb5444c8e82d1d3401874c4, State: Running, Role: LEADER
I20260806 03:55:34.980474 9335 consensus_queue.cc:237] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } }
I20260806 03:55:34.981196 8601 catalog_manager.cc:5495] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to bbb5dd0f1cb5444c8e82d1d3401874c4 (127.12.15.66). New cstate: current_term: 1 leader_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bbb5dd0f1cb5444c8e82d1d3401874c4" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 36977 } health_report { overall_health: HEALTHY } } }
I20260806 03:55:34.986912 9323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:34.984842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35334 (local address 127.12.15.66:36977)
0806 03:55:34.984877 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:34.984879 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:34.984928 (+ 49us) server_negotiation.cc:407] Connection header received
0806 03:55:34.984986 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:34.984987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:34.985001 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:34.985070 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:55:34.985183 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.985625 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.986336 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:34.986506 (+ 170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:34.986552 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:34.986650 (+ 98us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:55:34.986727 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:55:34.986791 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:34.986822 (+ 31us) server_negotiation.cc:299] Negotiation successful
0806 03:55:34.986828 (+ 6us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:55:34.988410 8826 raft_consensus.cc:1261] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 1 FOLLOWER]: Refusing update from remote peer bbb5dd0f1cb5444c8e82d1d3401874c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:34.988410 9239 raft_consensus.cc:1261] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 1 FOLLOWER]: Refusing update from remote peer bbb5dd0f1cb5444c8e82d1d3401874c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:34.988689 9339 consensus_queue.cc:1035] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0007c42ba500424a98d588db2499291a" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 38165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:34.988804 9335 consensus_queue.cc:1035] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e156670f24df485bbb593ea570a0484a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 42421 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:34.991080 9343 mvcc.cc:205] Tried to move back new op lower bound from 7315409039310467072 to 7315409039280607232. Current Snapshot: MvccSnapshot[applied={T|T < 7315409039310467072}]
I20260806 03:55:34.991786 9345 mvcc.cc:205] Tried to move back new op lower bound from 7315409039310467072 to 7315409039280607232. Current Snapshot: MvccSnapshot[applied={T|T < 7315409039310467072}]
I20260806 03:55:34.991899 9346 mvcc.cc:205] Tried to move back new op lower bound from 7315409039310467072 to 7315409039280607232. Current Snapshot: MvccSnapshot[applied={T|T < 7315409039310467072}]
I20260806 03:55:34.993398 8597 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:34.993477 8597 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:34.994143 8597 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9aa77b266514e4ba63c06512aef35e5: Sending DeleteTablet for 3 replicas of tablet d06751829cf6421eb84595cf4d2829c1
I20260806 03:55:34.994489 9209 tablet_service.cc:1514] Processing DeleteTablet for tablet d06751829cf6421eb84595cf4d2829c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:34 UTC) from {username='slave'} at 127.0.0.1:38550
I20260806 03:55:34.994526 9012 tablet_service.cc:1514] Processing DeleteTablet for tablet d06751829cf6421eb84595cf4d2829c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:34 UTC) from {username='slave'} at 127.0.0.1:35332
I20260806 03:55:34.994728 8794 tablet_service.cc:1514] Processing DeleteTablet for tablet d06751829cf6421eb84595cf4d2829c1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:34 UTC) from {username='slave'} at 127.0.0.1:48412
I20260806 03:55:34.994735 9362 tablet_replica.cc:330] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: stopping tablet replica
I20260806 03:55:34.994830 9362 raft_consensus.cc:2229] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:55:34.994902 9363 tablet_replica.cc:330] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a: stopping tablet replica
I20260806 03:55:34.994936 9362 raft_consensus.cc:2258] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:34.994992 9363 raft_consensus.cc:2229] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:34.995083 9363 raft_consensus.cc:2258] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:34.995365 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 8696
I20260806 03:55:34.995395 9362 ts_tablet_manager.cc:1905] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:34.995494 9363 ts_tablet_manager.cc:1905] T d06751829cf6421eb84595cf4d2829c1 P 0007c42ba500424a98d588db2499291a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:34.996925 9362 ts_tablet_manager.cc:1918] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:34.996987 9362 log.cc:1196] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/d06751829cf6421eb84595cf4d2829c1
I20260806 03:55:34.997239 9362 ts_tablet_manager.cc:1939] T d06751829cf6421eb84595cf4d2829c1 P bbb5dd0f1cb5444c8e82d1d3401874c4: Deleting consensus metadata
I20260806 03:55:34.997411 9360 tablet_replica.cc:330] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: stopping tablet replica
I20260806 03:55:34.997493 9360 raft_consensus.cc:2229] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:34.997577 9360 raft_consensus.cc:2258] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:34.998558 9360 ts_tablet_manager.cc:1905] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:35.000219 9360 ts_tablet_manager.cc:1918] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:35.000283 9360 log.cc:1196] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/d06751829cf6421eb84595cf4d2829c1
I20260806 03:55:35.000512 9360 ts_tablet_manager.cc:1939] T d06751829cf6421eb84595cf4d2829c1 P e156670f24df485bbb593ea570a0484a: Deleting consensus metadata
I20260806 03:55:35.000913 8573 catalog_manager.cc:4841] TS bbb5dd0f1cb5444c8e82d1d3401874c4 (127.12.15.66:36977): tablet d06751829cf6421eb84595cf4d2829c1 (table test-table [id=34f2f53e0c3b402e948db823680f215f]) successfully deleted
I20260806 03:55:35.003629 8571 catalog_manager.cc:4841] TS e156670f24df485bbb593ea570a0484a (127.12.15.67:42421): tablet d06751829cf6421eb84595cf4d2829c1 (table test-table [id=34f2f53e0c3b402e948db823680f215f]) successfully deleted
W20260806 03:55:35.007256 8573 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:38165 (error 108)
I20260806 03:55:35.007531 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 8908
I20260806 03:55:35.007738 9318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:35.007601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:38165 (local address 127.0.0.1:48418)
0806 03:55:35.007659 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:35.007702 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:38165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260806 03:55:35.007853 8573 catalog_manager.cc:4584] TS 0007c42ba500424a98d588db2499291a (127.12.15.65:38165): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d06751829cf6421eb84595cf4d2829c1: Network error: Client connection negotiation failed: client connection to 127.12.15.65:38165: connect: Connection refused (error 111)
I20260806 03:55:35.016531 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 9122
I20260806 03:55:35.030918 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 8548
2026-08-06T03:55:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (668 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:35 dist-test-slave-bqcl krb5kdc[9369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:35 dist-test-slave-bqcl krb5kdc[9369](info): set up 2 sockets
Aug 06 03:55:35 dist-test-slave-bqcl krb5kdc[9369](info): commencing operation
krb5kdc: starting...
W20260806 03:55:36.159375 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:55:36.274727 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:42639
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:46139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:42639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:36.354347 9387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:36.354548 9387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:36.354576 9387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:36.355695 9387 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:36.355736 9387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:36.355754 9387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:36.355769 9387 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:36.355784 9387 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:36.357196 9387 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:46139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:42639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:42639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:36.357427 9387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:36.357636 9387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:36.362363 9393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:36.362378 9398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:36.362380 9395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:36.362648 9387 server_base.cc:1034] running on GCE node
I20260806 03:55:36.362900 9387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:36.363155 9387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:36.364310 9387 hybrid_clock.cc:648] HybridClock initialized: now 1785988536364293 us; error 32 us; skew 500 ppm
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:36.366715 9387 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:55:36.367125 9387 webserver.cc:460] Webserver started at http://127.12.15.126:46093/ using document root <none> and password file <none>
I20260806 03:55:36.367313 9387 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:36.367360 9387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:36.367483 9387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:36.368454 9387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "fc3a8bab5e734e55ba6980cec1e3b99a"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "6122fe35ce0a7134f0317c6a7755d4e8"
server_key_iv: "20a92f0ed04ac478c77fb4ba277a0b8b"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.368784 9387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "fc3a8bab5e734e55ba6980cec1e3b99a"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "6122fe35ce0a7134f0317c6a7755d4e8"
server_key_iv: "20a92f0ed04ac478c77fb4ba277a0b8b"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.370011 9387 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:55:36.370698 9408 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.370844 9387 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.370926 9387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "fc3a8bab5e734e55ba6980cec1e3b99a"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "6122fe35ce0a7134f0317c6a7755d4e8"
server_key_iv: "20a92f0ed04ac478c77fb4ba277a0b8b"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.370986 9387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:36.379945 9387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:36.381880 9387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:36.382002 9387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:36.385841 9387 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:42639
I20260806 03:55:36.385874 9484 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:42639 every 8 connection(s)
I20260806 03:55:36.386205 9387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:36.387023 9486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:36.388821 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 9387
I20260806 03:55:36.388952 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:36.389148 12349 external_mini_cluster.cc:1428] Setting key 4b08d41fe4205b1eda1b56405d7ffec2
I20260806 03:55:36.389179 9486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a: Bootstrap starting.
I20260806 03:55:36.389819 9486 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:36.390113 9486 log.cc:824] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:36.390838 9486 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a: No bootstrap required, opened a new log
I20260806 03:55:36.392874 9486 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a8bab5e734e55ba6980cec1e3b99a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42639 } }
I20260806 03:55:36.393047 9486 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:36.393078 9486 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc3a8bab5e734e55ba6980cec1e3b99a, State: Initialized, Role: FOLLOWER
I20260806 03:55:36.393195 9486 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a8bab5e734e55ba6980cec1e3b99a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42639 } }
I20260806 03:55:36.393275 9486 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:36.393316 9486 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:36.393362 9486 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:36.394001 9486 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a8bab5e734e55ba6980cec1e3b99a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42639 } }
I20260806 03:55:36.394114 9486 leader_election.cc:302] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc3a8bab5e734e55ba6980cec1e3b99a; no voters:
I20260806 03:55:36.394299 9486 leader_election.cc:288] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:36.394378 9493 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:36.394536 9493 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [term 1 LEADER]: Becoming Leader. State: Replica: fc3a8bab5e734e55ba6980cec1e3b99a, State: Running, Role: LEADER
I20260806 03:55:36.394554 9486 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:36.394629 9493 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a8bab5e734e55ba6980cec1e3b99a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42639 } }
I20260806 03:55:36.395023 9494 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc3a8bab5e734e55ba6980cec1e3b99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a8bab5e734e55ba6980cec1e3b99a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42639 } } }
I20260806 03:55:36.395021 9495 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc3a8bab5e734e55ba6980cec1e3b99a. Latest consensus state: current_term: 1 leader_uuid: "fc3a8bab5e734e55ba6980cec1e3b99a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc3a8bab5e734e55ba6980cec1e3b99a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 42639 } } }
I20260806 03:55:36.395135 9494 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:36.395141 9495 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:36.395363 9497 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:36.395834 9497 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:36.397382 9497 catalog_manager.cc:1347] Generated new cluster ID: 74840ca126cf44879a9d91c2bc5019f9
I20260806 03:55:36.397436 9497 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:36.399744 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.389674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45758 (local address 127.12.15.126:42639)
0806 03:55:36.389849 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:36.389857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:36.389891 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:55:36.390430 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:36.390437 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:36.390489 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:36.390609 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:36.391360 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.392240 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.393269 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.393735 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.393816 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.396602 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:36.396606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:36.396606 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:36.396618 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:36.398403 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.398711 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.398714 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.398715 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.398802 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.398972 (+ 170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.398973 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.398973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.399196 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:36.399325 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:36.399442 (+ 117us) server_negotiation.cc:299] Negotiation successful
0806 03:55:36.399579 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":76,"threads_started":1}
I20260806 03:55:36.435976 9497 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:36.436698 9497 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:36.443261 9497 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a: Generated new TSK 0
I20260806 03:55:36.443439 9497 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.12.15.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
Entry for principal kudu/127.12.15.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab.
I20260806 03:55:36.474963 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42639
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:46139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:36.555141 9525 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:36.555339 9525 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:36.555368 9525 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:36.556483 9525 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:36.556525 9525 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:36.556559 9525 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:55:36.557983 9525 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:46139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.65.keytab
--principal=kudu/127.12.15.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9525
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:36.558239 9525 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:36.558457 9525 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:36.563237 9540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:36.563308 9533 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:36.563287 9532 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:36.563402 9525 server_base.cc:1034] running on GCE node
I20260806 03:55:36.563714 9525 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:36.563946 9525 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:36.565088 9525 hybrid_clock.cc:648] HybridClock initialized: now 1785988536565080 us; error 33 us; skew 500 ppm
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:36.567647 9525 init.cc:377] Logged in from keytab as kudu/127.12.15.65@KRBTEST.COM (short username kudu)
I20260806 03:55:36.568084 9525 webserver.cc:460] Webserver started at http://127.12.15.65:35573/ using document root <none> and password file <none>
I20260806 03:55:36.568295 9525 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:36.568339 9525 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:36.568454 9525 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:36.569442 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "b2c2f7a5a648430f8fa32ec327b96be7"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "02c1c0eb278cdbc0cdc5ba9c00c5ca86"
server_key_iv: "3c07a2a8c7ec535e59eb8e7b7a911e76"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.569761 9525 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "b2c2f7a5a648430f8fa32ec327b96be7"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "02c1c0eb278cdbc0cdc5ba9c00c5ca86"
server_key_iv: "3c07a2a8c7ec535e59eb8e7b7a911e76"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.571087 9525 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:55:36.571801 9548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.571990 9525 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260806 03:55:36.572090 9525 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "b2c2f7a5a648430f8fa32ec327b96be7"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "02c1c0eb278cdbc0cdc5ba9c00c5ca86"
server_key_iv: "3c07a2a8c7ec535e59eb8e7b7a911e76"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.572146 9525 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:36.587162 9525 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:36.588893 9525 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:36.589002 9525 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:36.589190 9525 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:36.589475 9525 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:36.589509 9525 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.589531 9525 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:36.589555 9525 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.595219 9525 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:37969
I20260806 03:55:36.595299 9728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:37969 every 8 connection(s)
I20260806 03:55:36.595577 9525 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:55:36.600116 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 9525
I20260806 03:55:36.600238 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:55:36.600426 12349 external_mini_cluster.cc:1428] Setting key 28ebeac10da6f1eae7ef90b62aefe0ac
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.65@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:36.604913 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.596249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:34131 (local address 127.12.15.126:42639)
0806 03:55:36.596293 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:36.596295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:36.596535 (+ 240us) server_negotiation.cc:407] Connection header received
0806 03:55:36.597052 (+ 517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:36.597053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:36.597065 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:36.597120 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:36.598074 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.598623 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.599605 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.599827 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.599885 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.602091 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:36.602092 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:36.602093 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:36.602100 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:36.603659 (+ 1559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.604037 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.604039 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.604039 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.604104 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.604309 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.604310 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.604310 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.604440 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:36.604504 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:36.604736 (+ 232us) server_negotiation.cc:299] Negotiation successful
0806 03:55:36.604825 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:55:36.604987 9732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.596237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42639 (local address 127.12.15.65:34131)
0806 03:55:36.596411 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:36.596448 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:36.596948 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:36.597194 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:36.597198 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:36.597770 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:36.597967 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.597973 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.598710 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.598711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.599513 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.599516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.599899 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.599900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.600045 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.600622 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:36.600630 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:36.602001 (+ 1371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:36.603740 (+ 1739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:36.603746 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:36.603748 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:36.603964 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:36.604161 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:36.604161 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:36.604162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:36.604255 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:36.604495 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:36.604498 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:36.604613 (+ 115us) client_negotiation.cc:769] Sending connection context
0806 03:55:36.604693 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:55:36.604814 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":86,"threads_started":1}
I20260806 03:55:36.605510 9730 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42639
I20260806 03:55:36.605607 9730 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:36.605795 9730 heartbeater.cc:511] Master 127.12.15.126:42639 requested a full tablet report, sending...
I20260806 03:55:36.606420 9429 ts_manager.cc:194] Registered new tserver with Master: b2c2f7a5a648430f8fa32ec327b96be7 (127.12.15.65:37969)
I20260806 03:55:36.607308 9429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.65@KRBTEST.COM'} at 127.12.15.65:34131
WARNING: no policy specified for kudu/127.12.15.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
Entry for principal kudu/127.12.15.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab.
I20260806 03:55:36.628439 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42639
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:46139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:36.719641 9737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:36.719904 9737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:36.719939 9737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:36.721599 9737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:36.721660 9737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:36.721696 9737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:55:36.723872 9737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:46139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.66.keytab
--principal=kudu/127.12.15.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9737
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:36.724145 9737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:36.724433 9737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:36.731171 9742 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:36.731242 9748 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:36.731199 9743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:36.731449 9737 server_base.cc:1034] running on GCE node
I20260806 03:55:36.731637 9737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:36.731869 9737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:36.733007 9737 hybrid_clock.cc:648] HybridClock initialized: now 1785988536732989 us; error 35 us; skew 500 ppm
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:36.735481 9737 init.cc:377] Logged in from keytab as kudu/127.12.15.66@KRBTEST.COM (short username kudu)
I20260806 03:55:36.735905 9737 webserver.cc:460] Webserver started at http://127.12.15.66:33871/ using document root <none> and password file <none>
I20260806 03:55:36.736090 9737 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:36.736129 9737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:36.736225 9737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:36.737190 9737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "c0b67de1b4d44ee090d7af0a2d410e11"
server_key_iv: "732ad4c497df70fda452ea2740e98f5d"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.737483 9737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "c0b67de1b4d44ee090d7af0a2d410e11"
server_key_iv: "732ad4c497df70fda452ea2740e98f5d"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.738771 9737 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:36.739652 9756 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.739820 9737 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:36.739905 9737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "c0b67de1b4d44ee090d7af0a2d410e11"
server_key_iv: "732ad4c497df70fda452ea2740e98f5d"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.739976 9737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:36.755777 9737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:36.757587 9737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:36.757720 9737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:36.757932 9737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:36.758286 9737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:36.758325 9737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.758354 9737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:36.758378 9737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.763964 9737 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:41493
I20260806 03:55:36.764050 9933 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:41493 every 8 connection(s)
I20260806 03:55:36.764333 9737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:55:36.765308 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 9737
I20260806 03:55:36.765408 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:55:36.765579 12349 external_mini_cluster.cc:1428] Setting key ea9c57cb9efe64cabafd8520076b243b
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.66@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:36.773398 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.764996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:48567 (local address 127.12.15.126:42639)
0806 03:55:36.765038 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:36.765040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:36.765248 (+ 208us) server_negotiation.cc:407] Connection header received
0806 03:55:36.765860 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:36.765861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:36.765873 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:36.765940 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:36.766981 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.767472 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.768459 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.768650 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.768715 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.770903 (+ 2188us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:36.770904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:36.770904 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:36.770911 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:36.772124 (+ 1213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.772481 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.772482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.772483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.772543 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.772768 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.772769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.772769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.772886 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:36.772943 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:36.773214 (+ 271us) server_negotiation.cc:299] Negotiation successful
0806 03:55:36.773306 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:55:36.773555 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.764974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42639 (local address 127.12.15.66:48567)
0806 03:55:36.765122 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:36.765159 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:36.765741 (+ 582us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:36.766014 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:36.766018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:36.766693 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:36.766889 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.766894 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.767554 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.767555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.768380 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.768382 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.768719 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.768720 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.768851 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.769382 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:36.769391 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:36.770815 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:36.772198 (+ 1383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:36.772205 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:36.772207 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:36.772416 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:36.772595 (+ 179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:36.772596 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:36.772596 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:36.772716 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:36.772943 (+ 227us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:36.772950 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:36.773084 (+ 134us) client_negotiation.cc:769] Sending connection context
0806 03:55:36.773191 (+ 107us) client_negotiation.cc:240] Negotiation successful
0806 03:55:36.773347 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":75,"threads_started":1}
I20260806 03:55:36.774196 9934 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42639
I20260806 03:55:36.774304 9934 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:36.774497 9934 heartbeater.cc:511] Master 127.12.15.126:42639 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.12.15.67@KRBTEST.COM; defaulting to no policy
I20260806 03:55:36.774901 9429 ts_manager.cc:194] Registered new tserver with Master: ee2d4ca47e9b48a6b57c15f11aa7d850 (127.12.15.66:41493)
I20260806 03:55:36.775265 9429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.66@KRBTEST.COM'} at 127.12.15.66:48567
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
Entry for principal kudu/127.12.15.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab.
I20260806 03:55:36.790622 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42639
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:46139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:36.869956 9943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:36.870185 9943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:36.870214 9943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:36.871351 9943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:36.871392 9943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:36.871415 9943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:55:36.872768 9943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:46139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.67.keytab
--principal=kudu/127.12.15.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:42639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:36.872967 9943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:36.873167 9943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:36.877904 9951 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:36.877882 9953 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:36.877893 9957 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:36.878252 9943 server_base.cc:1034] running on GCE node
I20260806 03:55:36.878440 9943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:36.878639 9943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:36.879774 9943 hybrid_clock.cc:648] HybridClock initialized: now 1785988536879768 us; error 32 us; skew 500 ppm
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:36.882266 9943 init.cc:377] Logged in from keytab as kudu/127.12.15.67@KRBTEST.COM (short username kudu)
I20260806 03:55:36.882700 9943 webserver.cc:460] Webserver started at http://127.12.15.67:35553/ using document root <none> and password file <none>
I20260806 03:55:36.882905 9943 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:36.882946 9943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:36.883042 9943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:36.883944 9943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "cb316cd393324829a0fc0c76e024783a"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "9a38bb7985e4bb62b674061cf807ac5a"
server_key_iv: "9b7efcb49075c00ac855897faa4bdaa4"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.884243 9943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "cb316cd393324829a0fc0c76e024783a"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "9a38bb7985e4bb62b674061cf807ac5a"
server_key_iv: "9b7efcb49075c00ac855897faa4bdaa4"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.885421 9943 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:55:36.886116 9969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.886294 9943 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:36.886379 9943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "cb316cd393324829a0fc0c76e024783a"
format_stamp: "Formatted at 2026-08-06 03:55:36 on dist-test-slave-bqcl"
server_key: "9a38bb7985e4bb62b674061cf807ac5a"
server_key_iv: "9b7efcb49075c00ac855897faa4bdaa4"
server_key_version: "encryptionkey@0"
I20260806 03:55:36.886431 9943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:36.903946 9943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:36.905894 9943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:36.905997 9943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:36.906203 9943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:36.906471 9943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:36.906502 9943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.906520 9943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:36.906531 9943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:36.911793 9943 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:43751
I20260806 03:55:36.911832 10142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:43751 every 8 connection(s)
I20260806 03:55:36.912109 9943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:55:36.915136 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 9943
I20260806 03:55:36.915218 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:55:36.915352 12349 external_mini_cluster.cc:1428] Setting key b0129153afce91489c5e2c36d22d8670
Aug 06 03:55:36 dist-test-slave-bqcl krb5kdc[9369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988536, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.67@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:36.920609 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.912780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:33871 (local address 127.12.15.126:42639)
0806 03:55:36.912828 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:36.912831 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:36.913029 (+ 198us) server_negotiation.cc:407] Connection header received
0806 03:55:36.913484 (+ 455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:36.913486 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:36.913495 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:36.913535 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:36.914447 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.914938 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.916182 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.916402 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.916450 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.918372 (+ 1922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:36.918373 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:36.918374 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:36.918380 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:36.919479 (+ 1099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.919810 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.919811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.919811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.919859 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.920142 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.920143 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.920143 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.920235 (+ 92us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:36.920280 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:36.920491 (+ 211us) server_negotiation.cc:299] Negotiation successful
0806 03:55:36.920547 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:55:36.920831 10145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.912762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:42639 (local address 127.12.15.67:33871)
0806 03:55:36.912920 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:36.912954 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:36.913396 (+ 442us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:36.913609 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:36.913612 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:36.914189 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:36.914379 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.914384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.915019 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.915020 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.916099 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.916102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.916461 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.916461 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.916568 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.916989 (+ 421us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:55:36.916996 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:36.918296 (+ 1300us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:36.919545 (+ 1249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:36.919551 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:36.919553 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:36.919754 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:36.919951 (+ 197us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:55:36.919952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:55:36.919953 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:55:36.920085 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:55:36.920283 (+ 198us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:36.920286 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:36.920395 (+ 109us) client_negotiation.cc:769] Sending connection context
0806 03:55:36.920478 (+ 83us) client_negotiation.cc:240] Negotiation successful
0806 03:55:36.920608 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260806 03:55:36.921258 10143 heartbeater.cc:347] Connected to a master server at 127.12.15.126:42639
I20260806 03:55:36.921350 10143 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:36.921530 10143 heartbeater.cc:511] Master 127.12.15.126:42639 requested a full tablet report, sending...
I20260806 03:55:36.921880 9429 ts_manager.cc:194] Registered new tserver with Master: cb316cd393324829a0fc0c76e024783a (127.12.15.67:43751)
I20260806 03:55:36.922292 9429 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.12.15.67@KRBTEST.COM'} at 127.12.15.67:33871
I20260806 03:55:36.926904 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:55:36.933375 9491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.928111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45766 (local address 127.12.15.126:42639)
0806 03:55:36.928150 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:36.928152 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:36.928269 (+ 117us) server_negotiation.cc:407] Connection header received
0806 03:55:36.928327 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:36.928328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:36.928335 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:36.928382 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:36.928884 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.929325 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.930032 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.930229 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.930273 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.931436 (+ 1163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:36.931436 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:36.931437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:36.931441 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:36.932460 (+ 1019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.932745 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.932746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.932746 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.932788 (+ 42us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:36.932913 (+ 125us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:36.932913 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:36.932914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:36.933034 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:36.933106 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:36.933262 (+ 156us) server_negotiation.cc:299] Negotiation successful
0806 03:55:36.933316 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:55:36.934881 9429 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:55:36.936434 9429 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:55:36.942284 10158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.938383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:43751 (local address 127.0.0.1:38796)
0806 03:55:36.938493 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:36.938531 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:36.938595 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:36.938994 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:36.938998 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:36.939000 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:36.939060 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.939061 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.940026 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.940027 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.940968 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.940970 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.941932 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.941933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.942016 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.942033 (+ 17us) client_negotiation.cc:769] Sending connection context
0806 03:55:36.942117 (+ 84us) client_negotiation.cc:240] Negotiation successful
0806 03:55:36.942195 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"thread_start_us":63,"threads_started":1}
I20260806 03:55:36.942678 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.938652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.12.15.67:43751)
0806 03:55:36.938846 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:36.938850 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:36.938860 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:55:36.938894 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:36.938896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:36.938911 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:36.938983 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:36.939201 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.939937 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.941082 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.941813 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.941922 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.941984 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:36.942243 (+ 259us) server_negotiation.cc:299] Negotiation successful
0806 03:55:36.942287 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":112,"threads_started":1}
I20260806 03:55:36.942968 10159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.938388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:37969 (local address 127.0.0.1:51152)
0806 03:55:36.938547 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:36.938557 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:36.938611 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:36.939297 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:36.939298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:36.939300 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:36.939331 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.939333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.940628 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.940629 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.941708 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.941711 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.942636 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.942637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.942703 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.942708 (+ 5us) client_negotiation.cc:769] Sending connection context
0806 03:55:36.942789 (+ 81us) client_negotiation.cc:240] Negotiation successful
0806 03:55:36.942840 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":64,"threads_started":1}
I20260806 03:55:36.943220 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.938879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.12.15.65:37969)
0806 03:55:36.939115 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:36.939119 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:36.939130 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:55:36.939170 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:36.939172 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:36.939187 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:36.939410 (+ 223us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:36.939481 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.940527 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.941808 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.942549 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.942619 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.942691 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:36.943091 (+ 400us) server_negotiation.cc:299] Negotiation successful
0806 03:55:36.943139 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":86,"threads_started":1}
I20260806 03:55:36.943240 10161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.938390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:41493 (local address 127.0.0.1:60314)
0806 03:55:36.938814 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:36.938824 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:36.938882 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:36.939160 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:36.939161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:36.939164 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:36.939218 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.939220 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.940570 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.940571 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.941695 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:36.941698 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.942948 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.942949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:36.943017 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.943021 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:36.943146 (+ 125us) client_negotiation.cc:240] Negotiation successful
0806 03:55:36.943183 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":33,"thread_start_us":70,"threads_started":1}
I20260806 03:55:36.943375 10041 tablet_service.cc:1467] Processing CreateTablet for tablet 33ff2c12cd354c748fc059bc31c4380b (DEFAULT_TABLE table=test-table [id=7c4580a55b58449d874f14fd0ecdb9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:36.943732 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:36.938646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60314 (local address 127.12.15.66:41493)
0806 03:55:36.938827 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:36.938832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:36.938876 (+ 44us) server_negotiation.cc:407] Connection header received
0806 03:55:36.938966 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:36.938968 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:36.938987 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:36.939067 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:36.939303 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.940430 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.941793 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:36.942765 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:36.942847 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:36.942909 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:36.943606 (+ 697us) server_negotiation.cc:299] Negotiation successful
0806 03:55:36.943653 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":85,"threads_started":1}
I20260806 03:55:36.943766 10041 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ff2c12cd354c748fc059bc31c4380b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:36.943904 9613 tablet_service.cc:1467] Processing CreateTablet for tablet 33ff2c12cd354c748fc059bc31c4380b (DEFAULT_TABLE table=test-table [id=7c4580a55b58449d874f14fd0ecdb9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:36.944236 9613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ff2c12cd354c748fc059bc31c4380b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:36.944285 9833 tablet_service.cc:1467] Processing CreateTablet for tablet 33ff2c12cd354c748fc059bc31c4380b (DEFAULT_TABLE table=test-table [id=7c4580a55b58449d874f14fd0ecdb9f9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:36.944552 9833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 33ff2c12cd354c748fc059bc31c4380b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:36.947124 10166 tablet_bootstrap.cc:492] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Bootstrap starting.
I20260806 03:55:36.947448 10167 tablet_bootstrap.cc:492] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Bootstrap starting.
I20260806 03:55:36.947438 10169 tablet_bootstrap.cc:492] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7: Bootstrap starting.
I20260806 03:55:36.948006 10167 tablet_bootstrap.cc:654] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:36.948043 10169 tablet_bootstrap.cc:654] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:36.948043 10166 tablet_bootstrap.cc:654] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:36.948460 10167 log.cc:824] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:36.948609 10169 log.cc:824] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:36.948604 10166 log.cc:824] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:36.949196 10167 tablet_bootstrap.cc:492] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: No bootstrap required, opened a new log
I20260806 03:55:36.949271 10167 ts_tablet_manager.cc:1397] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:36.949354 10169 tablet_bootstrap.cc:492] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7: No bootstrap required, opened a new log
I20260806 03:55:36.949420 10169 ts_tablet_manager.cc:1397] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:36.949612 10166 tablet_bootstrap.cc:492] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: No bootstrap required, opened a new log
I20260806 03:55:36.949687 10166 ts_tablet_manager.cc:1397] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260806 03:55:36.950901 10169 raft_consensus.cc:348] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:36.951087 10169 raft_consensus.cc:374] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:36.951139 10169 raft_consensus.cc:729] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2c2f7a5a648430f8fa32ec327b96be7, State: Initialized, Role: FOLLOWER
I20260806 03:55:36.951268 10169 consensus_queue.cc:260] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:36.951283 10167 raft_consensus.cc:348] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:36.951468 10167 raft_consensus.cc:374] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:36.951530 10167 raft_consensus.cc:729] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee2d4ca47e9b48a6b57c15f11aa7d850, State: Initialized, Role: FOLLOWER
I20260806 03:55:36.951536 10169 ts_tablet_manager.cc:1428] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:55:36.951660 9730 heartbeater.cc:503] Master 127.12.15.126:42639 was elected leader, sending a full tablet report...
I20260806 03:55:36.951655 10167 consensus_queue.cc:260] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:36.951627 10166 raft_consensus.cc:348] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:36.951784 10166 raft_consensus.cc:374] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:36.951835 10166 raft_consensus.cc:729] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb316cd393324829a0fc0c76e024783a, State: Initialized, Role: FOLLOWER
I20260806 03:55:36.951964 10167 ts_tablet_manager.cc:1428] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:55:36.952013 9934 heartbeater.cc:503] Master 127.12.15.126:42639 was elected leader, sending a full tablet report...
I20260806 03:55:36.951985 10166 consensus_queue.cc:260] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:36.952282 10143 heartbeater.cc:503] Master 127.12.15.126:42639 was elected leader, sending a full tablet report...
I20260806 03:55:36.952512 10166 ts_tablet_manager.cc:1428] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260806 03:55:37.014894 9936 tablet.cc:2369] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:55:37.096266 9731 tablet.cc:2369] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:55:37.162655 10144 tablet.cc:2369] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:55:37.165781 10178 raft_consensus.cc:482] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:37.165895 10178 raft_consensus.cc:504] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:37.166275 10178 leader_election.cc:288] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb316cd393324829a0fc0c76e024783a (127.12.15.67:43751), b2c2f7a5a648430f8fa32ec327b96be7 (127.12.15.65:37969)
I20260806 03:55:37.169363 10177 raft_consensus.cc:482] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:37.169466 10177 raft_consensus.cc:504] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:37.169963 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.166569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:52609 (local address 127.12.15.67:43751)
0806 03:55:37.166610 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.166611 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.166736 (+ 125us) server_negotiation.cc:407] Connection header received
0806 03:55:37.166784 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.166785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.166793 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.166833 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:37.166962 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.167649 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.169067 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.169628 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.169701 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.169710 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.169857 (+ 147us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.169896 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:55:37.170166 10165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.166514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:45683 (local address 127.12.15.65:37969)
0806 03:55:37.166574 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.166576 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.166587 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:55:37.166678 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.166679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.166691 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.166738 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:37.166896 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.167642 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.169067 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.169628 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.169701 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.169710 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.170034 (+ 324us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.170077 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:55:37.169999 10181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.166461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:43751 (local address 127.12.15.66:52609)
0806 03:55:37.166661 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.166670 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.166730 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.166845 (+ 115us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.166845 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.166846 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:37.166904 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.166905 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.167730 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.167731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.168992 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.168995 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.169702 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.169703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.169780 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.169784 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.169841 (+ 57us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.169883 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"mutex_wait_us":23,"thread_start_us":135,"threads_started":1}
I20260806 03:55:37.170004 9937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.166451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:37969 (local address 127.12.15.66:45683)
0806 03:55:37.166522 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.166533 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.166577 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.166755 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.166756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.166758 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:37.166827 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.166829 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.167730 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.167731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.168980 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.168984 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.169746 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.169748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.169827 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.169830 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.169879 (+ 49us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.169920 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":64}
I20260806 03:55:37.170686 9648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ff2c12cd354c748fc059bc31c4380b" candidate_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" is_pre_election: true
I20260806 03:55:37.170815 9648 raft_consensus.cc:2454] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee2d4ca47e9b48a6b57c15f11aa7d850 in term 0.
I20260806 03:55:37.170835 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ff2c12cd354c748fc059bc31c4380b" candidate_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb316cd393324829a0fc0c76e024783a" is_pre_election: true
I20260806 03:55:37.171057 10069 raft_consensus.cc:2454] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee2d4ca47e9b48a6b57c15f11aa7d850 in term 0.
I20260806 03:55:37.171267 9759 leader_election.cc:302] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb316cd393324829a0fc0c76e024783a, ee2d4ca47e9b48a6b57c15f11aa7d850; no voters:
I20260806 03:55:37.171355 10177 leader_election.cc:288] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb316cd393324829a0fc0c76e024783a (127.12.15.67:43751), ee2d4ca47e9b48a6b57c15f11aa7d850 (127.12.15.66:41493)
I20260806 03:55:37.171404 10178 raft_consensus.cc:2781] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:55:37.171453 10178 raft_consensus.cc:482] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:55:37.171474 10178 raft_consensus.cc:3037] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:37.172356 10178 raft_consensus.cc:504] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:37.172479 10178 leader_election.cc:288] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [CANDIDATE]: Term 1 election: Requested vote from peers cb316cd393324829a0fc0c76e024783a (127.12.15.67:43751), b2c2f7a5a648430f8fa32ec327b96be7 (127.12.15.65:37969)
I20260806 03:55:37.172650 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ff2c12cd354c748fc059bc31c4380b" candidate_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb316cd393324829a0fc0c76e024783a"
I20260806 03:55:37.172674 9648 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ff2c12cd354c748fc059bc31c4380b" candidate_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b2c2f7a5a648430f8fa32ec327b96be7"
I20260806 03:55:37.172720 10069 raft_consensus.cc:3037] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:37.172745 9648 raft_consensus.cc:3037] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:37.173508 10163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.170432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:47403 (local address 127.12.15.67:43751)
0806 03:55:37.170465 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.170467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.170473 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:55:37.170492 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.170493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.170500 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.170563 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:37.170708 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.171299 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.172623 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.173115 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.173160 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.173167 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.173418 (+ 251us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.173451 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:55:37.173586 10069 raft_consensus.cc:2454] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee2d4ca47e9b48a6b57c15f11aa7d850 in term 1.
I20260806 03:55:37.173627 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.170218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:39653 (local address 127.12.15.66:41493)
0806 03:55:37.170258 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.170260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.170587 (+ 327us) server_negotiation.cc:407] Connection header received
0806 03:55:37.170673 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.170674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.170684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.170712 (+ 28us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:37.171244 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.171749 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.172800 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.173275 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.173323 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.173330 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.173534 (+ 204us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.173570 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260806 03:55:37.173695 10183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.169945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:43751 (local address 127.12.15.65:47403)
0806 03:55:37.170171 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.170181 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.170232 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.170568 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.170569 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.170572 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:37.170631 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.170633 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.171401 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.171402 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.172525 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.172528 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.173226 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.173227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.173300 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.173304 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.173571 (+ 267us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.173616 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":8,"thread_start_us":57,"threads_started":1}
I20260806 03:55:37.173792 9759 leader_election.cc:302] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: cb316cd393324829a0fc0c76e024783a, ee2d4ca47e9b48a6b57c15f11aa7d850; no voters:
I20260806 03:55:37.173831 10185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.170289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:41493 (local address 127.12.15.65:39653)
0806 03:55:37.170528 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.170537 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.170611 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.170767 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.170767 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.170769 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:37.171139 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.171142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.171877 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.171878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.172709 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.172711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.173389 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.173390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.173455 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.173459 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.173703 (+ 244us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.173747 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":67,"threads_started":1}
I20260806 03:55:37.173877 10178 raft_consensus.cc:2781] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:37.174055 10178 raft_consensus.cc:686] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 1 LEADER]: Becoming Leader. State: Replica: ee2d4ca47e9b48a6b57c15f11aa7d850, State: Running, Role: LEADER
I20260806 03:55:37.174168 10178 consensus_queue.cc:237] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } }
I20260806 03:55:37.174371 9862 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ff2c12cd354c748fc059bc31c4380b" candidate_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" is_pre_election: true
I20260806 03:55:37.174985 9428 catalog_manager.cc:5495] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 reported cstate change: term changed from 0 to 1, leader changed from <none> to ee2d4ca47e9b48a6b57c15f11aa7d850 (127.12.15.66). New cstate: current_term: 1 leader_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee2d4ca47e9b48a6b57c15f11aa7d850" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 41493 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 } health_report { overall_health: UNKNOWN } } }
I20260806 03:55:37.175240 9648 raft_consensus.cc:2454] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee2d4ca47e9b48a6b57c15f11aa7d850 in term 1.
I20260806 03:55:37.175417 10069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "33ff2c12cd354c748fc059bc31c4380b" candidate_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb316cd393324829a0fc0c76e024783a" is_pre_election: true
I20260806 03:55:37.175504 10069 raft_consensus.cc:2379] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b2c2f7a5a648430f8fa32ec327b96be7 in current term 1: Already voted for candidate ee2d4ca47e9b48a6b57c15f11aa7d850 in this term.
I20260806 03:55:37.175770 9550 leader_election.cc:302] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b2c2f7a5a648430f8fa32ec327b96be7; no voters: cb316cd393324829a0fc0c76e024783a, ee2d4ca47e9b48a6b57c15f11aa7d850
I20260806 03:55:37.181989 10177 raft_consensus.cc:2726] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260806 03:55:37.223038 10162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.220849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60318 (local address 127.12.15.66:41493)
0806 03:55:37.220905 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.220907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.220915 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:55:37.220952 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.220953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.220972 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.221033 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:55:37.221140 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.221618 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.222388 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.222564 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.222608 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.222748 (+ 140us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:55:37.222852 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:55:37.222909 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.222922 (+ 13us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.222938 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260806 03:55:37.224841 10069 raft_consensus.cc:1261] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 1 FOLLOWER]: Refusing update from remote peer ee2d4ca47e9b48a6b57c15f11aa7d850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:37.224866 9648 raft_consensus.cc:1261] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 1 FOLLOWER]: Refusing update from remote peer ee2d4ca47e9b48a6b57c15f11aa7d850: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:37.225172 10186 consensus_queue.cc:1035] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb316cd393324829a0fc0c76e024783a" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 43751 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:37.225327 10178 consensus_queue.cc:1035] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b2c2f7a5a648430f8fa32ec327b96be7" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37969 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:37.227643 10191 mvcc.cc:205] Tried to move back new op lower bound from 7315409048470134784 to 7315409048265826304. Current Snapshot: MvccSnapshot[applied={T|T < 7315409048470134784}]
I20260806 03:55:37.227766 10195 mvcc.cc:205] Tried to move back new op lower bound from 7315409048470134784 to 7315409048265826304. Current Snapshot: MvccSnapshot[applied={T|T < 7315409048470134784}]
I20260806 03:55:37.228307 10194 mvcc.cc:205] Tried to move back new op lower bound from 7315409048470134784 to 7315409048265826304. Current Snapshot: MvccSnapshot[applied={T|T < 7315409048470134784}]
I20260806 03:55:37.230424 9429 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:37.230509 9429 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:37.231184 9429 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fc3a8bab5e734e55ba6980cec1e3b99a: Sending DeleteTablet for 3 replicas of tablet 33ff2c12cd354c748fc059bc31c4380b
I20260806 03:55:37.231508 9613 tablet_service.cc:1514] Processing DeleteTablet for tablet 33ff2c12cd354c748fc059bc31c4380b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:37 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:51152
I20260806 03:55:37.231503 10041 tablet_service.cc:1514] Processing DeleteTablet for tablet 33ff2c12cd354c748fc059bc31c4380b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:37 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:38796
I20260806 03:55:37.231506 9833 tablet_service.cc:1514] Processing DeleteTablet for tablet 33ff2c12cd354c748fc059bc31c4380b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:37 UTC) from {username='kudu', principal='kudu/127.12.15.126@KRBTEST.COM'} at 127.0.0.1:60314
I20260806 03:55:37.231751 10202 tablet_replica.cc:330] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: stopping tablet replica
I20260806 03:55:37.231846 10202 raft_consensus.cc:2229] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:55:37.231966 10202 raft_consensus.cc:2258] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:37.232082 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 9525
I20260806 03:55:37.232129 10205 tablet_replica.cc:330] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7: stopping tablet replica
I20260806 03:55:37.232210 10205 raft_consensus.cc:2229] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:37.232282 10205 raft_consensus.cc:2258] T 33ff2c12cd354c748fc059bc31c4380b P b2c2f7a5a648430f8fa32ec327b96be7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:37.232395 10203 tablet_replica.cc:330] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: stopping tablet replica
I20260806 03:55:37.232460 10203 raft_consensus.cc:2229] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:37.232527 10203 raft_consensus.cc:2258] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:37.242782 10203 ts_tablet_manager.cc:1905] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:37.244395 10203 ts_tablet_manager.cc:1918] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:37.244472 10203 log.cc:1196] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/33ff2c12cd354c748fc059bc31c4380b
I20260806 03:55:37.244753 10203 ts_tablet_manager.cc:1939] T 33ff2c12cd354c748fc059bc31c4380b P cb316cd393324829a0fc0c76e024783a: Deleting consensus metadata
I20260806 03:55:37.245244 9410 catalog_manager.cc:4841] TS cb316cd393324829a0fc0c76e024783a (127.12.15.67:43751): tablet 33ff2c12cd354c748fc059bc31c4380b (table test-table [id=7c4580a55b58449d874f14fd0ecdb9f9]) successfully deleted
I20260806 03:55:37.246443 10202 ts_tablet_manager.cc:1905] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:37.247681 10202 ts_tablet_manager.cc:1918] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:37.247752 10202 log.cc:1196] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/33ff2c12cd354c748fc059bc31c4380b
I20260806 03:55:37.247959 10202 ts_tablet_manager.cc:1939] T 33ff2c12cd354c748fc059bc31c4380b P ee2d4ca47e9b48a6b57c15f11aa7d850: Deleting consensus metadata
I20260806 03:55:37.248342 9412 catalog_manager.cc:4841] TS ee2d4ca47e9b48a6b57c15f11aa7d850 (127.12.15.66:41493): tablet 33ff2c12cd354c748fc059bc31c4380b (table test-table [id=7c4580a55b58449d874f14fd0ecdb9f9]) successfully deleted
W20260806 03:55:37.251341 9411 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:37969 (error 108)
I20260806 03:55:37.251583 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 9737
I20260806 03:55:37.253654 10161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.253467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:37969 (local address 127.0.0.1:51166)
0806 03:55:37.253570 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.253620 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:37969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260806 03:55:37.254717 9411 catalog_manager.cc:4584] TS b2c2f7a5a648430f8fa32ec327b96be7 (127.12.15.65:37969): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 33ff2c12cd354c748fc059bc31c4380b: Network error: Client connection negotiation failed: client connection to 127.12.15.65:37969: connect: Connection refused (error 111)
I20260806 03:55:37.262825 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 9943
I20260806 03:55:37.276436 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 9387
2026-08-06T03:55:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2250 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-06T03:55:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:37Z Disabled control of system clock
I20260806 03:55:37.311519 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:39697
--webserver_interface=127.12.15.126
--webserver_port=0
--builtin_ntp_servers=127.12.15.84:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:39697
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:37.390607 10216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:37.390822 10216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:37.390841 10216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:37.391937 10216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:37.391969 10216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:37.391979 10216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:37.391990 10216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:37.392001 10216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:37.393277 10216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:39697
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:39697
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10216
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:37.393462 10216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:37.393703 10216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:37.398294 10225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:37.398272 10229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:37.398316 10216 server_base.cc:1034] running on GCE node
W20260806 03:55:37.398326 10226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:37.398849 10216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:37.399117 10216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:37.400266 10216 hybrid_clock.cc:648] HybridClock initialized: now 1785988537400245 us; error 37 us; skew 500 ppm
I20260806 03:55:37.401441 10216 webserver.cc:460] Webserver started at http://127.12.15.126:38399/ using document root <none> and password file <none>
I20260806 03:55:37.401698 10216 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:37.401746 10216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:37.401882 10216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:37.402988 10216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "908ba4972fc249dd90922de73d0d4c2a"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "e31dd1dc517e0dfc22a1e7db0891cf10"
server_key_iv: "dbbff893ea10fe619adf592f67901906"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.403347 10216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "908ba4972fc249dd90922de73d0d4c2a"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "e31dd1dc517e0dfc22a1e7db0891cf10"
server_key_iv: "dbbff893ea10fe619adf592f67901906"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.404593 10216 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:55:37.405233 10241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.405368 10216 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:37.405442 10216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "908ba4972fc249dd90922de73d0d4c2a"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "e31dd1dc517e0dfc22a1e7db0891cf10"
server_key_iv: "dbbff893ea10fe619adf592f67901906"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.405504 10216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:37.415392 10216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:37.415652 10216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:37.415771 10216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:37.419528 10216 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:39697
I20260806 03:55:37.419559 10322 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:39697 every 8 connection(s)
I20260806 03:55:37.419862 10216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:37.420819 10323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:37.423209 10323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a: Bootstrap starting.
I20260806 03:55:37.423784 10323 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:37.424064 10323 log.cc:824] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:37.424731 10323 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a: No bootstrap required, opened a new log
I20260806 03:55:37.425639 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 10216
I20260806 03:55:37.425756 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:37.425935 12349 external_mini_cluster.cc:1428] Setting key c937fbf67b5427d6088bcdf122bbe53a
I20260806 03:55:37.426512 10323 raft_consensus.cc:348] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ba4972fc249dd90922de73d0d4c2a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39697 } }
I20260806 03:55:37.426699 10323 raft_consensus.cc:374] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:37.426728 10323 raft_consensus.cc:729] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 908ba4972fc249dd90922de73d0d4c2a, State: Initialized, Role: FOLLOWER
I20260806 03:55:37.426851 10323 consensus_queue.cc:260] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ba4972fc249dd90922de73d0d4c2a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39697 } }
I20260806 03:55:37.426910 10323 raft_consensus.cc:388] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:37.426932 10323 raft_consensus.cc:482] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:37.426977 10323 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:37.427480 10323 raft_consensus.cc:504] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ba4972fc249dd90922de73d0d4c2a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39697 } }
I20260806 03:55:37.427569 10323 leader_election.cc:302] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 908ba4972fc249dd90922de73d0d4c2a; no voters:
I20260806 03:55:37.427744 10323 leader_election.cc:288] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:37.427788 10330 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:37.427902 10330 raft_consensus.cc:686] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [term 1 LEADER]: Becoming Leader. State: Replica: 908ba4972fc249dd90922de73d0d4c2a, State: Running, Role: LEADER
I20260806 03:55:37.428002 10323 sys_catalog.cc:564] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:37.427982 10330 consensus_queue.cc:237] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ba4972fc249dd90922de73d0d4c2a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39697 } }
I20260806 03:55:37.428328 10332 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "908ba4972fc249dd90922de73d0d4c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ba4972fc249dd90922de73d0d4c2a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39697 } } }
I20260806 03:55:37.428426 10332 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:37.428314 10330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 908ba4972fc249dd90922de73d0d4c2a. Latest consensus state: current_term: 1 leader_uuid: "908ba4972fc249dd90922de73d0d4c2a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "908ba4972fc249dd90922de73d0d4c2a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 39697 } } }
I20260806 03:55:37.428601 10330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:37.428719 10338 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:37.429462 10338 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:37.431106 10338 catalog_manager.cc:1347] Generated new cluster ID: 2220c8e39e7c412a822fbd9140fab7d6
I20260806 03:55:37.431171 10338 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:37.432360 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.426514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59482 (local address 127.12.15.126:39697)
0806 03:55:37.426747 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.426756 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.426793 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:55:37.426915 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.426922 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.426994 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.427129 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:37.427358 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.429070 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.430260 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.430837 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.430961 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.431674 (+ 713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:37.431678 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:37.431678 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:37.431860 (+ 182us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:37.431895 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:37.431906 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:37.431967 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.432071 (+ 104us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.432150 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
I20260806 03:55:37.452004 10338 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:37.452559 10338 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:37.457448 10338 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a: Generated new TSK 0
I20260806 03:55:37.457654 10338 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:37.463828 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.65:0
--local_ip_for_outbound_sockets=127.12.15.65
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39697
--builtin_ntp_servers=127.12.15.84:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:37.545204 10358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:37.545424 10358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:37.545445 10358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:37.546608 10358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:37.546644 10358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:37.546669 10358 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.65
I20260806 03:55:37.547969 10358 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.12.15.65
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.65
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:37.548185 10358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:37.548455 10358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:37.553349 10365 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:37.553357 10369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:37.553443 10367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:37.554018 10358 server_base.cc:1034] running on GCE node
I20260806 03:55:37.554239 10358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:37.554466 10358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:37.555615 10358 hybrid_clock.cc:648] HybridClock initialized: now 1785988537555599 us; error 32 us; skew 500 ppm
I20260806 03:55:37.556766 10358 webserver.cc:460] Webserver started at http://127.12.15.65:45343/ using document root <none> and password file <none>
I20260806 03:55:37.556996 10358 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:37.557055 10358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:37.557188 10358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:37.558310 10358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/data/instance:
uuid: "d4c786e2f8304d56930c374a9c48abd2"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "b5a3bdb563fd05bb5710d9787f461f4c"
server_key_iv: "c3c8767c8565a06f5a77045e02d27958"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.558640 10358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance:
uuid: "d4c786e2f8304d56930c374a9c48abd2"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "b5a3bdb563fd05bb5710d9787f461f4c"
server_key_iv: "c3c8767c8565a06f5a77045e02d27958"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.559911 10358 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:55:37.560695 10378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.560853 10358 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:37.560933 10358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/wal
uuid: "d4c786e2f8304d56930c374a9c48abd2"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "b5a3bdb563fd05bb5710d9787f461f4c"
server_key_iv: "c3c8767c8565a06f5a77045e02d27958"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.560994 10358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:37.569842 10358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:37.570107 10358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:37.570266 10358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:37.570468 10358 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:37.570770 10358 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:37.570803 10358 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.570832 10358 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:37.570855 10358 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.576503 10358 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.65:37689
I20260806 03:55:37.576570 10559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.65:37689 every 8 connection(s)
I20260806 03:55:37.576864 10358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/data/info.pb
I20260806 03:55:37.579248 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 10358
I20260806 03:55:37.579349 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-0/wal/instance
I20260806 03:55:37.579515 12349 external_mini_cluster.cc:1428] Setting key 9f89979f49d72f917d3af352556c3566
I20260806 03:55:37.580428 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.66:0
--local_ip_for_outbound_sockets=127.12.15.66
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39697
--builtin_ntp_servers=127.12.15.84:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260806 03:55:37.582067 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.577553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:58225 (local address 127.12.15.126:39697)
0806 03:55:37.577615 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.577617 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.577840 (+ 223us) server_negotiation.cc:407] Connection header received
0806 03:55:37.578362 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.578362 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.578374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.578431 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:37.579070 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.579622 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.580732 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.580938 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.580992 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.581798 (+ 806us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:37.581800 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:37.581800 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:37.581807 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:37.581824 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:37.581828 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:37.581875 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.581958 (+ 83us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.581989 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260806 03:55:37.582285 10562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.577526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:39697 (local address 127.12.15.65:58225)
0806 03:55:37.577725 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.577770 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.578280 (+ 510us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.578487 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.578490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.578498 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:37.579005 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.579009 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.579709 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.579710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.580651 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.580655 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.581016 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.581017 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.581147 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.581660 (+ 513us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:37.581660 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:37.581689 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:37.581713 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:37.581881 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:37.581886 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:37.581889 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.581954 (+ 65us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.582066 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260806 03:55:37.582764 10560 heartbeater.cc:347] Connected to a master server at 127.12.15.126:39697
I20260806 03:55:37.582866 10560 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:37.583058 10560 heartbeater.cc:511] Master 127.12.15.126:39697 requested a full tablet report, sending...
I20260806 03:55:37.583606 10264 ts_manager.cc:194] Registered new tserver with Master: d4c786e2f8304d56930c374a9c48abd2 (127.12.15.65:37689)
I20260806 03:55:37.584447 10264 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.65:58225
W20260806 03:55:37.664839 10564 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:37.665053 10564 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:37.665074 10564 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:37.666253 10564 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:37.666289 10564 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:37.666313 10564 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.66
I20260806 03:55:37.667676 10564 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.12.15.66
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10564
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.66
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:37.667894 10564 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:37.668172 10564 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:37.673018 10579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:37.672988 10572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:37.672995 10576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:37.673259 10564 server_base.cc:1034] running on GCE node
I20260806 03:55:37.673455 10564 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:37.673693 10564 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:37.674831 10564 hybrid_clock.cc:648] HybridClock initialized: now 1785988537674816 us; error 32 us; skew 500 ppm
I20260806 03:55:37.676086 10564 webserver.cc:460] Webserver started at http://127.12.15.66:42553/ using document root <none> and password file <none>
I20260806 03:55:37.676311 10564 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:37.676369 10564 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:37.676524 10564 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:37.677567 10564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/data/instance:
uuid: "9dccf71231ef439aa4937d36bf175889"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "bd96fd2ba11e9be475ccafff0e7cd765"
server_key_iv: "f4acf2c9dd080e9879863194d9db1678"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.677894 10564 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance:
uuid: "9dccf71231ef439aa4937d36bf175889"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "bd96fd2ba11e9be475ccafff0e7cd765"
server_key_iv: "f4acf2c9dd080e9879863194d9db1678"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.679153 10564 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:55:37.679926 10588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.680083 10564 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260806 03:55:37.680167 10564 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/wal
uuid: "9dccf71231ef439aa4937d36bf175889"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "bd96fd2ba11e9be475ccafff0e7cd765"
server_key_iv: "f4acf2c9dd080e9879863194d9db1678"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.680233 10564 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:37.704450 10564 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:37.704763 10564 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:37.704895 10564 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:37.705108 10564 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:37.705454 10564 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:37.705490 10564 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.705519 10564 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:37.705546 10564 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.711400 10564 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.66:37087
I20260806 03:55:37.711463 10773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.66:37087 every 8 connection(s)
I20260806 03:55:37.711762 10564 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/data/info.pb
I20260806 03:55:37.716493 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.712450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.66:60243 (local address 127.12.15.126:39697)
0806 03:55:37.712529 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.712531 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.712737 (+ 206us) server_negotiation.cc:407] Connection header received
0806 03:55:37.713221 (+ 484us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.713222 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.713232 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.713283 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:37.713905 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.714436 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.715363 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.715536 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.715576 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.716314 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:37.716315 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:37.716315 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:37.716318 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:37.716329 (+ 11us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:37.716332 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:37.716361 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.716418 (+ 57us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.716435 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260806 03:55:37.716702 10780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.712421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:39697 (local address 127.12.15.66:60243)
0806 03:55:37.712620 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.712654 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.713142 (+ 488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.713343 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.713346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.713354 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:37.713843 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.713848 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.714513 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.714514 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.715299 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.715302 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.715588 (+ 286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.715589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.715693 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.716201 (+ 508us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:37.716202 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:37.716231 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:37.716254 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:37.716370 (+ 116us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:37.716373 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:37.716375 (+ 2us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.716417 (+ 42us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.716496 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":96,"threads_started":1}
I20260806 03:55:37.716944 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 10564
I20260806 03:55:37.717048 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/wal/instance
I20260806 03:55:37.717193 10774 heartbeater.cc:347] Connected to a master server at 127.12.15.126:39697
I20260806 03:55:37.717239 12349 external_mini_cluster.cc:1428] Setting key 97bcd7018b34b1ce5fe685d52456fd4f
I20260806 03:55:37.717278 10774 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:37.717453 10774 heartbeater.cc:511] Master 127.12.15.126:39697 requested a full tablet report, sending...
I20260806 03:55:37.717777 10264 ts_manager.cc:194] Registered new tserver with Master: 9dccf71231ef439aa4937d36bf175889 (127.12.15.66:37087)
I20260806 03:55:37.718142 10264 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.66:60243
I20260806 03:55:37.718152 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.12.15.67:0
--local_ip_for_outbound_sockets=127.12.15.67
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39697
--builtin_ntp_servers=127.12.15.84:43667
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260806 03:55:37.800354 10781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:37.800566 10781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:37.800587 10781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:37.801673 10781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:37.801707 10781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:37.801729 10781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.12.15.67
I20260806 03:55:37.803102 10781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:43667
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.12.15.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.12.15.67
--webserver_port=0
--tserver_master_addrs=127.12.15.126:39697
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.12.15.67
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:37.803298 10781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:37.803545 10781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:37.808347 10791 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:37.808341 10796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:37.808432 10781 server_base.cc:1034] running on GCE node
W20260806 03:55:37.808341 10788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:37.808926 10781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:37.809192 10781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:37.810328 10781 hybrid_clock.cc:648] HybridClock initialized: now 1785988537810315 us; error 32 us; skew 500 ppm
I20260806 03:55:37.811581 10781 webserver.cc:460] Webserver started at http://127.12.15.67:34567/ using document root <none> and password file <none>
I20260806 03:55:37.811793 10781 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:37.811841 10781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:37.811964 10781 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:37.812994 10781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/data/instance:
uuid: "ca81c95c48c8460abdabb07a8897c3da"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "900e6d3173dd38dcbc20e99ba8e6aeab"
server_key_iv: "832a706660b82edce45ddfda2d28a31c"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.813325 10781 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance:
uuid: "ca81c95c48c8460abdabb07a8897c3da"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "900e6d3173dd38dcbc20e99ba8e6aeab"
server_key_iv: "832a706660b82edce45ddfda2d28a31c"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.814553 10781 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:55:37.815207 10807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.815353 10781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:37.815438 10781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/wal
uuid: "ca81c95c48c8460abdabb07a8897c3da"
format_stamp: "Formatted at 2026-08-06 03:55:37 on dist-test-slave-bqcl"
server_key: "900e6d3173dd38dcbc20e99ba8e6aeab"
server_key_iv: "832a706660b82edce45ddfda2d28a31c"
server_key_version: "encryptionkey@0"
I20260806 03:55:37.815496 10781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:37.826067 10781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:37.826336 10781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:37.826448 10781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:37.826638 10781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260806 03:55:37.826917 10781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260806 03:55:37.826951 10781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.826977 10781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260806 03:55:37.827003 10781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:37.832458 10781 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.67:38493
I20260806 03:55:37.832517 11001 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.67:38493 every 8 connection(s)
I20260806 03:55:37.832788 10781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/data/info.pb
I20260806 03:55:37.833473 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 10781
I20260806 03:55:37.833587 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/wal/instance
I20260806 03:55:37.833755 12349 external_mini_cluster.cc:1428] Setting key ba24471b59f712f6960ac3b182cc8481
I20260806 03:55:37.837833 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.833474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.67:42335 (local address 127.12.15.126:39697)
0806 03:55:37.833541 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.833543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.833815 (+ 272us) server_negotiation.cc:407] Connection header received
0806 03:55:37.834416 (+ 601us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.834417 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.834430 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.834475 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:37.835149 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.835642 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.836597 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.836789 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.836844 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.837584 (+ 740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:37.837586 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:37.837586 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:37.837590 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:37.837607 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:37.837611 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:37.837652 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.837724 (+ 72us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.837757 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:55:37.838029 11005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.833449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:39697 (local address 127.12.15.67:42335)
0806 03:55:37.833705 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.833742 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.834319 (+ 577us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.834542 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.834546 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.834556 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:55:37.835063 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.835068 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.835728 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.835729 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.836517 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.836519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.836861 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.836862 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.836994 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.837455 (+ 461us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0806 03:55:37.837456 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:55:37.837483 (+ 27us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0806 03:55:37.837506 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:55:37.837658 (+ 152us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:55:37.837664 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:55:37.837667 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.837728 (+ 61us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.837825 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":75,"threads_started":1}
I20260806 03:55:37.838407 11003 heartbeater.cc:347] Connected to a master server at 127.12.15.126:39697
I20260806 03:55:37.838483 11003 heartbeater.cc:464] Registering TS with master...
I20260806 03:55:37.838646 11003 heartbeater.cc:511] Master 127.12.15.126:39697 requested a full tablet report, sending...
I20260806 03:55:37.838948 10264 ts_manager.cc:194] Registered new tserver with Master: ca81c95c48c8460abdabb07a8897c3da (127.12.15.67:38493)
I20260806 03:55:37.839313 10264 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.12.15.67:42335
I20260806 03:55:37.845427 12349 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260806 03:55:37.849175 10328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.846744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59496 (local address 127.12.15.126:39697)
0806 03:55:37.846784 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.846786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.846884 (+ 98us) server_negotiation.cc:407] Connection header received
0806 03:55:37.846927 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.846927 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.846936 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.846982 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:37.847101 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.847542 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.848228 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.848402 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.848443 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.849018 (+ 575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:37.849019 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:37.849019 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:37.849023 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:37.849032 (+ 9us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:37.849035 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:37.849063 (+ 28us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.849103 (+ 40us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.849119 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:55:37.850584 10264 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59496:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260806 03:55:37.852141 10264 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260806 03:55:37.858345 11016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.854162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:37689 (local address 127.0.0.1:55326)
0806 03:55:37.854456 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.854498 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.854597 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.854820 (+ 223us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.854823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.854825 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:37.854891 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.854892 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.856200 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.856201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.857226 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.857228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.858022 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.858023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.858108 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.858148 (+ 40us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.858214 (+ 66us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.858269 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":66,"thread_start_us":73,"threads_started":1}
I20260806 03:55:37.858352 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.854347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55326 (local address 127.12.15.65:37689)
0806 03:55:37.854560 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.854565 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.854578 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:55:37.854698 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.854700 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.854718 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.854798 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:37.854973 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.856103 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.857297 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.857927 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.857999 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.858099 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.858231 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.858269 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":104,"threads_started":1}
I20260806 03:55:37.858487 11022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.854465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:37087 (local address 127.0.0.1:42392)
0806 03:55:37.854622 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.854631 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.854683 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.855004 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.855005 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.855007 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:37.855050 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.855052 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.856256 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.856257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.857461 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.857464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.858285 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.858285 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.858343 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.858346 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.858390 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.858425 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":94,"thread_start_us":80,"threads_started":1}
I20260806 03:55:37.858718 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.854288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42392 (local address 127.12.15.66:37087)
0806 03:55:37.854807 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.854812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.854821 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:55:37.854857 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.854859 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.854874 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.854938 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:37.855115 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.856188 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.857561 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.858075 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.858161 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.858249 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.858582 (+ 333us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.858632 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":88,"threads_started":1}
I20260806 03:55:37.858882 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.854282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38493 (local address 127.0.0.1:48938)
0806 03:55:37.854552 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:37.854561 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:37.854600 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:37.855320 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:37.855320 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:37.855322 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:37.855360 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.855362 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.856159 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.856159 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.857461 (+ 1302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:37.857464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.858603 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.858605 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:37.858667 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.858671 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:37.858718 (+ 47us) client_negotiation.cc:240] Negotiation successful
0806 03:55:37.858756 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":3,"thread_start_us":87,"threads_started":1}
I20260806 03:55:37.859115 11017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:37.854275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48938 (local address 127.12.15.67:38493)
0806 03:55:37.855185 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:37.855190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:37.855199 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:55:37.855229 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:37.855231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:37.855246 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:37.855310 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:37.855416 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.856081 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.857549 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:37.858236 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:37.858296 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:37.858372 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:37.859002 (+ 630us) server_negotiation.cc:299] Negotiation successful
0806 03:55:37.859041 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":874,"thread_start_us":87,"threads_started":1}
I20260806 03:55:37.859174 10455 tablet_service.cc:1467] Processing CreateTablet for tablet fd6fdf039068496fa4c25bc06d5ab4d2 (DEFAULT_TABLE table=test-table [id=02e293bee5a04352b47396a2209813a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:37.859411 10659 tablet_service.cc:1467] Processing CreateTablet for tablet fd6fdf039068496fa4c25bc06d5ab4d2 (DEFAULT_TABLE table=test-table [id=02e293bee5a04352b47396a2209813a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:37.859540 10455 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd6fdf039068496fa4c25bc06d5ab4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:37.859683 10659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd6fdf039068496fa4c25bc06d5ab4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:37.862008 11023 tablet_bootstrap.cc:492] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2: Bootstrap starting.
I20260806 03:55:37.862008 11024 tablet_bootstrap.cc:492] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Bootstrap starting.
I20260806 03:55:37.862697 11024 tablet_bootstrap.cc:654] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:37.862756 11023 tablet_bootstrap.cc:654] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:37.862720 10897 tablet_service.cc:1467] Processing CreateTablet for tablet fd6fdf039068496fa4c25bc06d5ab4d2 (DEFAULT_TABLE table=test-table [id=02e293bee5a04352b47396a2209813a8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260806 03:55:37.863016 11024 log.cc:824] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:37.863035 11023 log.cc:824] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:37.863054 10897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fd6fdf039068496fa4c25bc06d5ab4d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:37.863807 11024 tablet_bootstrap.cc:492] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: No bootstrap required, opened a new log
I20260806 03:55:37.863881 11024 ts_tablet_manager.cc:1397] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:55:37.863952 11023 tablet_bootstrap.cc:492] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2: No bootstrap required, opened a new log
I20260806 03:55:37.864018 11023 ts_tablet_manager.cc:1397] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:37.865971 11023 raft_consensus.cc:348] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:37.866147 11023 raft_consensus.cc:374] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:37.866196 11023 raft_consensus.cc:729] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4c786e2f8304d56930c374a9c48abd2, State: Initialized, Role: FOLLOWER
I20260806 03:55:37.866343 11027 tablet_bootstrap.cc:492] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Bootstrap starting.
I20260806 03:55:37.866325 11023 consensus_queue.cc:260] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:37.866364 11024 raft_consensus.cc:348] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:37.866504 11024 raft_consensus.cc:374] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:37.866554 11024 raft_consensus.cc:729] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9dccf71231ef439aa4937d36bf175889, State: Initialized, Role: FOLLOWER
I20260806 03:55:37.866585 11023 ts_tablet_manager.cc:1428] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:55:37.866662 11024 consensus_queue.cc:260] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:37.866732 10560 heartbeater.cc:503] Master 127.12.15.126:39697 was elected leader, sending a full tablet report...
I20260806 03:55:37.866904 11024 ts_tablet_manager.cc:1428] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260806 03:55:37.866973 10774 heartbeater.cc:503] Master 127.12.15.126:39697 was elected leader, sending a full tablet report...
I20260806 03:55:37.866979 11027 tablet_bootstrap.cc:654] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:37.867241 11027 log.cc:824] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:37.867959 11027 tablet_bootstrap.cc:492] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: No bootstrap required, opened a new log
I20260806 03:55:37.868029 11027 ts_tablet_manager.cc:1397] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260806 03:55:37.869786 11027 raft_consensus.cc:348] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:37.869899 11027 raft_consensus.cc:374] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:37.869918 11027 raft_consensus.cc:729] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ca81c95c48c8460abdabb07a8897c3da, State: Initialized, Role: FOLLOWER
I20260806 03:55:37.869992 11027 consensus_queue.cc:260] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:37.870208 11027 ts_tablet_manager.cc:1428] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260806 03:55:37.870237 11003 heartbeater.cc:503] Master 127.12.15.126:39697 was elected leader, sending a full tablet report...
W20260806 03:55:37.962306 10775 tablet.cc:2369] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:55:38.077561 10561 tablet.cc:2369] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260806 03:55:38.083302 11004 tablet.cc:2369] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260806 03:55:38.171242 11028 raft_consensus.cc:482] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260806 03:55:38.171403 11028 raft_consensus.cc:504] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:38.171778 11028 leader_election.cc:288] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ca81c95c48c8460abdabb07a8897c3da (127.12.15.67:38493), 9dccf71231ef439aa4937d36bf175889 (127.12.15.66:37087)
I20260806 03:55:38.174726 11017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:38.172043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:46147 (local address 127.12.15.67:38493)
0806 03:55:38.172094 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:38.172096 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:38.172207 (+ 111us) server_negotiation.cc:407] Connection header received
0806 03:55:38.172291 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:38.172292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:38.172300 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:38.172349 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:38.172490 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.172972 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.174008 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.174390 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.174454 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:38.174464 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:38.174617 (+ 153us) server_negotiation.cc:299] Negotiation successful
0806 03:55:38.174657 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:55:38.174736 11019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:38.172056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.12.15.65:35591 (local address 127.12.15.66:37087)
0806 03:55:38.172115 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:38.172117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:38.172201 (+ 84us) server_negotiation.cc:407] Connection header received
0806 03:55:38.172263 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:38.172264 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:38.172274 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:38.172320 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0806 03:55:38.172480 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.172960 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.174021 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.174414 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.174482 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:38.174491 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:38.174624 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 03:55:38.174669 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:55:38.174749 11034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:38.171957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.66:37087 (local address 127.12.15.65:35591)
0806 03:55:38.172110 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:38.172122 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:38.172202 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:38.172337 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:38.172338 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:38.172339 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:38.172419 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:38.172421 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.173038 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.173039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:38.173927 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:38.173929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.174491 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.174493 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:38.174565 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:38.174568 (+ 3us) client_negotiation.cc:769] Sending connection context
0806 03:55:38.174613 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:55:38.174656 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":79,"threads_started":1}
I20260806 03:55:38.174754 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:38.171957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.67:38493 (local address 127.12.15.65:46147)
0806 03:55:38.172127 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:38.172138 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:55:38.172228 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:55:38.172357 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:55:38.172357 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:55:38.172359 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0806 03:55:38.172436 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:38.172437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.173077 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.173078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:38.173913 (+ 835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:55:38.173916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.174472 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.174473 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:55:38.174551 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:38.174555 (+ 4us) client_negotiation.cc:769] Sending connection context
0806 03:55:38.174601 (+ 46us) client_negotiation.cc:240] Negotiation successful
0806 03:55:38.174648 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":34,"thread_start_us":60,"threads_started":1}
I20260806 03:55:38.175112 10702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd6fdf039068496fa4c25bc06d5ab4d2" candidate_uuid: "d4c786e2f8304d56930c374a9c48abd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dccf71231ef439aa4937d36bf175889" is_pre_election: true
I20260806 03:55:38.175304 10702 raft_consensus.cc:2454] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c786e2f8304d56930c374a9c48abd2 in term 0.
I20260806 03:55:38.175307 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd6fdf039068496fa4c25bc06d5ab4d2" candidate_uuid: "d4c786e2f8304d56930c374a9c48abd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca81c95c48c8460abdabb07a8897c3da" is_pre_election: true
I20260806 03:55:38.175441 10924 raft_consensus.cc:2454] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d4c786e2f8304d56930c374a9c48abd2 in term 0.
I20260806 03:55:38.175549 10379 leader_election.cc:302] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dccf71231ef439aa4937d36bf175889, d4c786e2f8304d56930c374a9c48abd2; no voters:
I20260806 03:55:38.175690 11028 raft_consensus.cc:2781] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260806 03:55:38.175761 11028 raft_consensus.cc:482] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260806 03:55:38.175784 11028 raft_consensus.cc:3037] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:38.176501 11028 raft_consensus.cc:504] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:38.176638 11028 leader_election.cc:288] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [CANDIDATE]: Term 1 election: Requested vote from peers ca81c95c48c8460abdabb07a8897c3da (127.12.15.67:38493), 9dccf71231ef439aa4937d36bf175889 (127.12.15.66:37087)
I20260806 03:55:38.176796 10702 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd6fdf039068496fa4c25bc06d5ab4d2" candidate_uuid: "d4c786e2f8304d56930c374a9c48abd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9dccf71231ef439aa4937d36bf175889"
I20260806 03:55:38.176793 10924 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fd6fdf039068496fa4c25bc06d5ab4d2" candidate_uuid: "d4c786e2f8304d56930c374a9c48abd2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ca81c95c48c8460abdabb07a8897c3da"
I20260806 03:55:38.176869 10702 raft_consensus.cc:3037] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:38.176872 10924 raft_consensus.cc:3037] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:38.177587 10924 raft_consensus.cc:2454] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c786e2f8304d56930c374a9c48abd2 in term 1.
I20260806 03:55:38.177596 10702 raft_consensus.cc:2454] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d4c786e2f8304d56930c374a9c48abd2 in term 1.
I20260806 03:55:38.177778 10379 leader_election.cc:302] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9dccf71231ef439aa4937d36bf175889, d4c786e2f8304d56930c374a9c48abd2; no voters:
I20260806 03:55:38.177871 11028 raft_consensus.cc:2781] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:38.178045 11028 raft_consensus.cc:686] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 1 LEADER]: Becoming Leader. State: Replica: d4c786e2f8304d56930c374a9c48abd2, State: Running, Role: LEADER
I20260806 03:55:38.178184 11028 consensus_queue.cc:237] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } }
I20260806 03:55:38.179080 10263 catalog_manager.cc:5495] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 reported cstate change: term changed from 0 to 1, leader changed from <none> to d4c786e2f8304d56930c374a9c48abd2 (127.12.15.65). New cstate: current_term: 1 leader_uuid: "d4c786e2f8304d56930c374a9c48abd2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d4c786e2f8304d56930c374a9c48abd2" member_type: VOTER last_known_addr { host: "127.12.15.65" port: 37689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 } health_report { overall_health: UNKNOWN } } }
I20260806 03:55:38.212311 11021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:38.210038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55336 (local address 127.12.15.65:37689)
0806 03:55:38.210106 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:38.210109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:38.210117 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:55:38.210182 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:38.210183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:38.210204 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:38.210268 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:55:38.210387 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.210857 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.211588 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:38.211780 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:38.211836 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:38.211978 (+ 142us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:55:38.212099 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:55:38.212189 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:38.212206 (+ 17us) server_negotiation.cc:299] Negotiation successful
0806 03:55:38.212230 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260806 03:55:38.214207 10702 raft_consensus.cc:1261] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 1 FOLLOWER]: Refusing update from remote peer d4c786e2f8304d56930c374a9c48abd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:38.214289 10924 raft_consensus.cc:1261] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 1 FOLLOWER]: Refusing update from remote peer d4c786e2f8304d56930c374a9c48abd2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:55:38.214594 11028 consensus_queue.cc:1035] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ca81c95c48c8460abdabb07a8897c3da" member_type: VOTER last_known_addr { host: "127.12.15.67" port: 38493 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:38.214741 11036 consensus_queue.cc:1035] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9dccf71231ef439aa4937d36bf175889" member_type: VOTER last_known_addr { host: "127.12.15.66" port: 37087 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:55:38.216930 11042 mvcc.cc:205] Tried to move back new op lower bound from 7315409052522336256 to 7315409052378451968. Current Snapshot: MvccSnapshot[applied={T|T < 7315409052522336256}]
I20260806 03:55:38.217892 11043 mvcc.cc:205] Tried to move back new op lower bound from 7315409052522336256 to 7315409052378451968. Current Snapshot: MvccSnapshot[applied={T|T < 7315409052522336256}]
I20260806 03:55:38.219245 10263 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:38.219324 10263 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59496:
table { table_name: "test-table" } modify_external_catalogs: true
I20260806 03:55:38.220018 10263 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 908ba4972fc249dd90922de73d0d4c2a: Sending DeleteTablet for 3 replicas of tablet fd6fdf039068496fa4c25bc06d5ab4d2
I20260806 03:55:38.220351 10659 tablet_service.cc:1514] Processing DeleteTablet for tablet fd6fdf039068496fa4c25bc06d5ab4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:38 UTC) from {username='slave'} at 127.0.0.1:42392
I20260806 03:55:38.220340 10455 tablet_service.cc:1514] Processing DeleteTablet for tablet fd6fdf039068496fa4c25bc06d5ab4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:38 UTC) from {username='slave'} at 127.0.0.1:55326
I20260806 03:55:38.220486 10897 tablet_service.cc:1514] Processing DeleteTablet for tablet fd6fdf039068496fa4c25bc06d5ab4d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-06 03:55:38 UTC) from {username='slave'} at 127.0.0.1:48938
I20260806 03:55:38.220578 11056 tablet_replica.cc:330] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2: stopping tablet replica
I20260806 03:55:38.220678 11056 raft_consensus.cc:2229] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 1 LEADER]: Raft consensus shutting down.
I20260806 03:55:38.220777 11056 raft_consensus.cc:2258] T fd6fdf039068496fa4c25bc06d5ab4d2 P d4c786e2f8304d56930c374a9c48abd2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:38.220942 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 10358
I20260806 03:55:38.230726 11057 tablet_replica.cc:330] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: stopping tablet replica
I20260806 03:55:38.230850 11057 raft_consensus.cc:2229] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:38.230952 11057 raft_consensus.cc:2258] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:38.236603 11055 tablet_replica.cc:330] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: stopping tablet replica
I20260806 03:55:38.236728 11055 raft_consensus.cc:2229] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260806 03:55:38.236829 11055 raft_consensus.cc:2258] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260806 03:55:38.237067 11057 ts_tablet_manager.cc:1905] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:38.237723 11055 ts_tablet_manager.cc:1905] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260806 03:55:38.238718 11057 ts_tablet_manager.cc:1918] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:38.238793 11057 log.cc:1196] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-2/wal/wals/fd6fdf039068496fa4c25bc06d5ab4d2
I20260806 03:55:38.239051 11057 ts_tablet_manager.cc:1939] T fd6fdf039068496fa4c25bc06d5ab4d2 P ca81c95c48c8460abdabb07a8897c3da: Deleting consensus metadata
I20260806 03:55:38.239491 10246 catalog_manager.cc:4841] TS ca81c95c48c8460abdabb07a8897c3da (127.12.15.67:38493): tablet fd6fdf039068496fa4c25bc06d5ab4d2 (table test-table [id=02e293bee5a04352b47396a2209813a8]) successfully deleted
I20260806 03:55:38.239579 11055 ts_tablet_manager.cc:1918] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260806 03:55:38.239668 11055 log.cc:1196] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Deleting WAL directory at /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1785988295328500-12349-0/minicluster-data/ts-1/wal/wals/fd6fdf039068496fa4c25bc06d5ab4d2
I20260806 03:55:38.239897 11055 ts_tablet_manager.cc:1939] T fd6fdf039068496fa4c25bc06d5ab4d2 P 9dccf71231ef439aa4937d36bf175889: Deleting consensus metadata
I20260806 03:55:38.240275 10242 catalog_manager.cc:4841] TS 9dccf71231ef439aa4937d36bf175889 (127.12.15.66:37087): tablet fd6fdf039068496fa4c25bc06d5ab4d2 (table test-table [id=02e293bee5a04352b47396a2209813a8]) successfully deleted
W20260806 03:55:38.241437 10243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.12.15.65:37689 (error 108)
I20260806 03:55:38.241920 11018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:38.241771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.65:37689 (local address 127.0.0.1:55346)
0806 03:55:38.241838 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0806 03:55:38.241889 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.65:37689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260806 03:55:38.241927 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 10564
W20260806 03:55:38.247380 10243 catalog_manager.cc:4584] TS d4c786e2f8304d56930c374a9c48abd2 (127.12.15.65:37689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fd6fdf039068496fa4c25bc06d5ab4d2: Network error: Client connection negotiation failed: client connection to 127.12.15.65:37689: connect: Connection refused (error 111)
I20260806 03:55:38.260315 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 10781
I20260806 03:55:38.268193 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 10216
2026-08-06T03:55:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (985 ms)
[----------] 6 tests from EncryptionPolicyTest (9912 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:38 dist-test-slave-bqcl krb5kdc[11063](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:38 dist-test-slave-bqcl krb5kdc[11063](info): set up 2 sockets
Aug 06 03:55:38 dist-test-slave-bqcl krb5kdc[11063](info): commencing operation
krb5kdc: starting...
W20260806 03:55:39.389724 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:39 dist-test-slave-bqcl krb5kdc[11063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:39.508664 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37397
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37397
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:39.589282 11077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:39.589488 11077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:39.589519 11077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:39.590735 11077 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:39.590790 11077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:39.590809 11077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:39.590826 11077 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:39.590840 11077 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:39.592353 11077 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37397
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37397
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:39.592609 11077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:39.592836 11077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:39.597662 11084 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:39.597678 11077 server_base.cc:1034] running on GCE node
W20260806 03:55:39.597643 11086 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:39.597636 11083 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:39.598146 11077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:39.598465 11077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:39.599637 11077 hybrid_clock.cc:648] HybridClock initialized: now 1785988539599616 us; error 29 us; skew 500 ppm
Aug 06 03:55:39 dist-test-slave-bqcl krb5kdc[11063](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988539, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:39.606257 11077 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:39.606694 11077 webserver.cc:460] Webserver started at http://127.0.0.1:45917/ using document root <none> and password file <none>
I20260806 03:55:39.606901 11077 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:39.606946 11077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:39.607053 11077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:39.608105 11077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "220862f6b1b34009ad79f480f6c69302"
format_stamp: "Formatted at 2026-08-06 03:55:39 on dist-test-slave-bqcl"
server_key: "5d5742d36c03cd36daaa04291843b3bd"
server_key_iv: "fb1a0baedf9d1c6fabd7ed9660fc9ed2"
server_key_version: "encryptionkey@0"
I20260806 03:55:39.608421 11077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "220862f6b1b34009ad79f480f6c69302"
format_stamp: "Formatted at 2026-08-06 03:55:39 on dist-test-slave-bqcl"
server_key: "5d5742d36c03cd36daaa04291843b3bd"
server_key_iv: "fb1a0baedf9d1c6fabd7ed9660fc9ed2"
server_key_version: "encryptionkey@0"
I20260806 03:55:39.609719 11077 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:39.610468 11098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:39.610627 11077 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:39.610713 11077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "220862f6b1b34009ad79f480f6c69302"
format_stamp: "Formatted at 2026-08-06 03:55:39 on dist-test-slave-bqcl"
server_key: "5d5742d36c03cd36daaa04291843b3bd"
server_key_iv: "fb1a0baedf9d1c6fabd7ed9660fc9ed2"
server_key_version: "encryptionkey@0"
I20260806 03:55:39.610770 11077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:39.619845 11077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:39.621589 11077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:39.621695 11077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:39.625418 11077 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37397
I20260806 03:55:39.625456 11179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37397 every 8 connection(s)
I20260806 03:55:39.625766 11077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:39.626622 11180 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:39.628863 11180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302: Bootstrap starting.
I20260806 03:55:39.629412 11180 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:39.629693 11180 log.cc:824] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:39.630370 11180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302: No bootstrap required, opened a new log
I20260806 03:55:39.631937 11180 raft_consensus.cc:348] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220862f6b1b34009ad79f480f6c69302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37397 } }
I20260806 03:55:39.632081 11180 raft_consensus.cc:374] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:39.632113 11180 raft_consensus.cc:729] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 220862f6b1b34009ad79f480f6c69302, State: Initialized, Role: FOLLOWER
I20260806 03:55:39.632242 11180 consensus_queue.cc:260] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220862f6b1b34009ad79f480f6c69302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37397 } }
I20260806 03:55:39.632303 11180 raft_consensus.cc:388] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:39.632332 11180 raft_consensus.cc:482] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:39.632380 11180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:39.632889 11180 raft_consensus.cc:504] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220862f6b1b34009ad79f480f6c69302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37397 } }
I20260806 03:55:39.632988 11180 leader_election.cc:302] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 220862f6b1b34009ad79f480f6c69302; no voters:
I20260806 03:55:39.633152 11180 leader_election.cc:288] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:39.633185 11183 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:39.633333 11183 raft_consensus.cc:686] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [term 1 LEADER]: Becoming Leader. State: Replica: 220862f6b1b34009ad79f480f6c69302, State: Running, Role: LEADER
I20260806 03:55:39.633406 11180 sys_catalog.cc:564] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:39.633477 11183 consensus_queue.cc:237] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220862f6b1b34009ad79f480f6c69302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37397 } }
I20260806 03:55:39.633553 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 11077
I20260806 03:55:39.633673 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:39.633874 12349 external_mini_cluster.cc:1428] Setting key 777d68f94629e71cf0802e0332699997
I20260806 03:55:39.634560 11185 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "220862f6b1b34009ad79f480f6c69302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220862f6b1b34009ad79f480f6c69302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37397 } } }
I20260806 03:55:39.634671 11185 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:39.634865 11207 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:39.634876 11186 sys_catalog.cc:455] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 220862f6b1b34009ad79f480f6c69302. Latest consensus state: current_term: 1 leader_uuid: "220862f6b1b34009ad79f480f6c69302" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "220862f6b1b34009ad79f480f6c69302" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37397 } } }
I20260806 03:55:39.635123 11186 sys_catalog.cc:458] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:39.635545 11207 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:39.637068 11207 catalog_manager.cc:1347] Generated new cluster ID: cb5f6eba4c6746c5a7741112f5c4d770
I20260806 03:55:39.637120 11207 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:55:39 dist-test-slave-bqcl krb5kdc[11063](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988539, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 03:55:39.645526 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:39.634552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44771 (local address 127.0.0.1:37397)
0806 03:55:39.634952 (+ 400us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:39.634962 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:39.635057 (+ 95us) server_negotiation.cc:407] Connection header received
0806 03:55:39.635177 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:39.635184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:39.635239 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:39.635357 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:39.636062 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:39.637359 (+ 1297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:39.638432 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:39.639025 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:39.639133 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:39.641905 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:39.641910 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:39.641910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:39.641924 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:39.643859 (+ 1935us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:39.644279 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:39.644282 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:39.644283 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:39.644412 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:39.644628 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:39.644629 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:39.644629 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:39.644935 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:39.645095 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:39.645176 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 03:55:39.645299 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":75,"threads_started":1}
I20260806 03:55:39.646602 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:39.650161 11207 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:39.650691 11207 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:39.654965 11207 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 220862f6b1b34009ad79f480f6c69302: Generated new TSK 0
I20260806 03:55:39.655058 11201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:39.647977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49567 (local address 127.0.0.1:37397)
0806 03:55:39.648012 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:39.648014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:39.648078 (+ 64us) server_negotiation.cc:407] Connection header received
0806 03:55:39.648129 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:39.648129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:39.648138 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:39.648204 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:39.648740 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:39.649506 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:39.650441 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:39.650770 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:39.650830 (+ 60us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:39.652181 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:39.652182 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:39.652182 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:39.652189 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:39.653665 (+ 1476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:39.654078 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:39.654079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:39.654079 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:39.654163 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:39.654402 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:39.654403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:39.654403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:39.654521 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:39.654606 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:39.654854 (+ 248us) server_negotiation.cc:299] Negotiation successful
0806 03:55:39.654919 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:55:39.655578 11207 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:39.661633 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 11077
W20260806 03:55:39.675961 11060 connection.cc:535] client connection to 127.0.0.1:37397 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37397): Connection reset by peer (error 104)
2026-08-06T03:55:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:39 dist-test-slave-bqcl krb5kdc[11220](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:39 dist-test-slave-bqcl krb5kdc[11220](info): set up 2 sockets
Aug 06 03:55:39 dist-test-slave-bqcl krb5kdc[11220](info): commencing operation
krb5kdc: starting...
W20260806 03:55:40.812268 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:40 dist-test-slave-bqcl krb5kdc[11220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988540, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:40.927151 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46367
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34861
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46367
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:41.009382 11240 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:41.009608 11240 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:41.009631 11240 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:41.010824 11240 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:41.010869 11240 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:41.010882 11240 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:41.010905 11240 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:41.010927 11240 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:41.012455 11240 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34861
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46367
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46367
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11240
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:41.012715 11240 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:41.012928 11240 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:55:41.017738 11240 server_base.cc:1034] running on GCE node
W20260806 03:55:41.017693 11246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:41.017685 11252 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:41.017693 11245 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:41.018213 11240 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:41.018489 11240 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:41.019646 11240 hybrid_clock.cc:648] HybridClock initialized: now 1785988541019632 us; error 28 us; skew 500 ppm
Aug 06 03:55:41 dist-test-slave-bqcl krb5kdc[11220](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988541, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:41.022073 11240 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:41.022527 11240 webserver.cc:460] Webserver started at http://127.0.0.1:43885/ using document root <none> and password file <none>
I20260806 03:55:41.022734 11240 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:41.022779 11240 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:41.022890 11240 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:41.023916 11240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "830f2318bc2e40c8a452010fb6cb688f"
format_stamp: "Formatted at 2026-08-06 03:55:41 on dist-test-slave-bqcl"
server_key: "83deb786d231cd8fe9e7e1a4ddcc5ed2"
server_key_iv: "35d45483efbafb90296b592bbc0dbf9c"
server_key_version: "encryptionkey@0"
I20260806 03:55:41.024240 11240 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "830f2318bc2e40c8a452010fb6cb688f"
format_stamp: "Formatted at 2026-08-06 03:55:41 on dist-test-slave-bqcl"
server_key: "83deb786d231cd8fe9e7e1a4ddcc5ed2"
server_key_iv: "35d45483efbafb90296b592bbc0dbf9c"
server_key_version: "encryptionkey@0"
I20260806 03:55:41.026207 11240 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260806 03:55:41.026918 11266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:41.027060 11240 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:41.027145 11240 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "830f2318bc2e40c8a452010fb6cb688f"
format_stamp: "Formatted at 2026-08-06 03:55:41 on dist-test-slave-bqcl"
server_key: "83deb786d231cd8fe9e7e1a4ddcc5ed2"
server_key_iv: "35d45483efbafb90296b592bbc0dbf9c"
server_key_version: "encryptionkey@0"
I20260806 03:55:41.027215 11240 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:41.046432 11240 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:41.048381 11240 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:41.048524 11240 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:41.052383 11240 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46367
I20260806 03:55:41.052421 11337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46367 every 8 connection(s)
I20260806 03:55:41.052758 11240 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:41.053664 11338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:41.055933 11338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f: Bootstrap starting.
I20260806 03:55:41.056455 11338 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:41.056689 11338 log.cc:824] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:41.057336 11338 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f: No bootstrap required, opened a new log
I20260806 03:55:41.058977 11338 raft_consensus.cc:348] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830f2318bc2e40c8a452010fb6cb688f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } }
I20260806 03:55:41.059116 11338 raft_consensus.cc:374] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:41.059144 11338 raft_consensus.cc:729] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 830f2318bc2e40c8a452010fb6cb688f, State: Initialized, Role: FOLLOWER
I20260806 03:55:41.059244 11338 consensus_queue.cc:260] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830f2318bc2e40c8a452010fb6cb688f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } }
I20260806 03:55:41.059293 11338 raft_consensus.cc:388] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:41.059312 11338 raft_consensus.cc:482] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:41.059341 11338 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:41.059825 11338 raft_consensus.cc:504] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830f2318bc2e40c8a452010fb6cb688f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } }
I20260806 03:55:41.059916 11338 leader_election.cc:302] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 830f2318bc2e40c8a452010fb6cb688f; no voters:
I20260806 03:55:41.060050 11338 leader_election.cc:288] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:41.060091 11343 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:41.060271 11338 sys_catalog.cc:564] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:41.060284 11343 raft_consensus.cc:686] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [term 1 LEADER]: Becoming Leader. State: Replica: 830f2318bc2e40c8a452010fb6cb688f, State: Running, Role: LEADER
I20260806 03:55:41.060446 11343 consensus_queue.cc:237] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830f2318bc2e40c8a452010fb6cb688f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } }
I20260806 03:55:41.060811 11345 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "830f2318bc2e40c8a452010fb6cb688f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830f2318bc2e40c8a452010fb6cb688f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } } }
I20260806 03:55:41.060891 11345 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:41.060931 11347 sys_catalog.cc:455] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 830f2318bc2e40c8a452010fb6cb688f. Latest consensus state: current_term: 1 leader_uuid: "830f2318bc2e40c8a452010fb6cb688f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "830f2318bc2e40c8a452010fb6cb688f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46367 } } }
I20260806 03:55:41.060982 11347 sys_catalog.cc:458] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:41.061148 11356 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:41.061712 11356 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:41.062211 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 11240
I20260806 03:55:41.062314 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:41.062485 12349 external_mini_cluster.cc:1428] Setting key a9f49dacf81be7a5c3cdcb8ef7e674f8
I20260806 03:55:41.063359 11356 catalog_manager.cc:1347] Generated new cluster ID: f297c7d87b6347a9b5892cdf75b8ceee
I20260806 03:55:41.063409 11356 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:41.068416 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:41.063111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34555 (local address 127.0.0.1:46367)
0806 03:55:41.063260 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:41.063270 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:41.063305 (+ 35us) server_negotiation.cc:407] Connection header received
0806 03:55:41.063418 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:41.063424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:41.063478 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:41.063621 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:41.063831 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:41.065265 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:41.066330 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:41.066937 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:41.067051 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:41.067722 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:41.067727 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:41.067727 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:41.067936 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:41.067980 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:41.067995 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:41.068085 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:41.068170 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 03:55:41.068201 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260806 03:55:41.069355 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:41.073988 11376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:41.070856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57453 (local address 127.0.0.1:46367)
0806 03:55:41.070890 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:41.070891 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:41.070899 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:55:41.070919 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:41.070920 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:41.070929 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:41.070985 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:41.071113 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:41.071816 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:41.072755 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:41.073069 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:41.073137 (+ 68us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:41.073755 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:41.073756 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:41.073757 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:41.073761 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:41.073777 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:41.073782 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:41.073840 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:41.073900 (+ 60us) server_negotiation.cc:299] Negotiation successful
0806 03:55:41.073913 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:55:41.093760 11356 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:41.094302 11356 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:41.108013 11356 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 830f2318bc2e40c8a452010fb6cb688f: Generated new TSK 0
I20260806 03:55:41.108228 11356 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:41.120122 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 11240
W20260806 03:55:41.134800 11218 connection.cc:535] client connection to 127.0.0.1:46367 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46367): Connection reset by peer (error 104)
2026-08-06T03:55:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1459 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:41 dist-test-slave-bqcl krb5kdc[11392](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:41 dist-test-slave-bqcl krb5kdc[11392](info): set up 2 sockets
Aug 06 03:55:41 dist-test-slave-bqcl krb5kdc[11392](info): commencing operation
krb5kdc: starting...
W20260806 03:55:42.252148 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.104s user 0.006s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:42 dist-test-slave-bqcl krb5kdc[11392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988542, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:42.371130 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41643
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:42.452724 11414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:42.452920 11414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:42.452942 11414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:42.454105 11414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:42.454173 11414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:42.454188 11414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:42.454213 11414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:42.454236 11414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:42.455753 11414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41643
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:42.455971 11414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:42.456182 11414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:42.460731 11423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:42.460721 11420 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:42.460909 11414 server_base.cc:1034] running on GCE node
W20260806 03:55:42.460731 11429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:42.461217 11414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:42.461509 11414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:42.462656 11414 hybrid_clock.cc:648] HybridClock initialized: now 1785988542462636 us; error 35 us; skew 500 ppm
Aug 06 03:55:42 dist-test-slave-bqcl krb5kdc[11392](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988542, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:42.465193 11414 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:42.465667 11414 webserver.cc:460] Webserver started at http://127.0.0.1:38393/ using document root <none> and password file <none>
I20260806 03:55:42.465891 11414 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:42.465951 11414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:42.466079 11414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:42.467134 11414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "91e3d02a77ba46e1ba73524b5f3cfcec"
format_stamp: "Formatted at 2026-08-06 03:55:42 on dist-test-slave-bqcl"
server_key: "466ad978bc173ea42ba816848e35f33f"
server_key_iv: "a3b0d75e433db0a81d10c0b8c3744cfe"
server_key_version: "encryptionkey@0"
I20260806 03:55:42.467473 11414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "91e3d02a77ba46e1ba73524b5f3cfcec"
format_stamp: "Formatted at 2026-08-06 03:55:42 on dist-test-slave-bqcl"
server_key: "466ad978bc173ea42ba816848e35f33f"
server_key_iv: "a3b0d75e433db0a81d10c0b8c3744cfe"
server_key_version: "encryptionkey@0"
I20260806 03:55:42.468793 11414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:55:42.469520 11440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:42.469708 11414 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:42.469813 11414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "91e3d02a77ba46e1ba73524b5f3cfcec"
format_stamp: "Formatted at 2026-08-06 03:55:42 on dist-test-slave-bqcl"
server_key: "466ad978bc173ea42ba816848e35f33f"
server_key_iv: "a3b0d75e433db0a81d10c0b8c3744cfe"
server_key_version: "encryptionkey@0"
I20260806 03:55:42.469882 11414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:42.473357 11414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:42.475229 11414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:42.475358 11414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:42.479291 11414 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34477
I20260806 03:55:42.479322 11518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34477 every 8 connection(s)
I20260806 03:55:42.479622 11414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:42.480536 11519 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:42.482846 11519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec: Bootstrap starting.
I20260806 03:55:42.483369 11519 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:42.483608 11519 log.cc:824] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:42.484217 11519 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec: No bootstrap required, opened a new log
I20260806 03:55:42.485776 11519 raft_consensus.cc:348] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e3d02a77ba46e1ba73524b5f3cfcec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34477 } }
I20260806 03:55:42.485910 11519 raft_consensus.cc:374] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:42.485932 11519 raft_consensus.cc:729] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 91e3d02a77ba46e1ba73524b5f3cfcec, State: Initialized, Role: FOLLOWER
I20260806 03:55:42.486055 11519 consensus_queue.cc:260] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e3d02a77ba46e1ba73524b5f3cfcec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34477 } }
I20260806 03:55:42.486162 11519 raft_consensus.cc:388] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:42.486204 11519 raft_consensus.cc:482] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:42.486209 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 11414
I20260806 03:55:42.486256 11519 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:42.486320 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:42.486503 12349 external_mini_cluster.cc:1428] Setting key 6c40f352963d148e01823caea41fd915
I20260806 03:55:42.486896 11519 raft_consensus.cc:504] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e3d02a77ba46e1ba73524b5f3cfcec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34477 } }
I20260806 03:55:42.487071 11519 leader_election.cc:302] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 91e3d02a77ba46e1ba73524b5f3cfcec; no voters:
I20260806 03:55:42.487224 11519 leader_election.cc:288] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:42.487419 11526 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:42.487473 11519 sys_catalog.cc:564] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:42.487655 11526 raft_consensus.cc:686] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [term 1 LEADER]: Becoming Leader. State: Replica: 91e3d02a77ba46e1ba73524b5f3cfcec, State: Running, Role: LEADER
I20260806 03:55:42.487783 11526 consensus_queue.cc:237] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e3d02a77ba46e1ba73524b5f3cfcec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34477 } }
I20260806 03:55:42.488138 11527 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "91e3d02a77ba46e1ba73524b5f3cfcec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e3d02a77ba46e1ba73524b5f3cfcec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34477 } } }
I20260806 03:55:42.488138 11528 sys_catalog.cc:455] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [sys.catalog]: SysCatalogTable state changed. Reason: New leader 91e3d02a77ba46e1ba73524b5f3cfcec. Latest consensus state: current_term: 1 leader_uuid: "91e3d02a77ba46e1ba73524b5f3cfcec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "91e3d02a77ba46e1ba73524b5f3cfcec" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34477 } } }
I20260806 03:55:42.488241 11527 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:42.488247 11528 sys_catalog.cc:458] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:42.488449 11531 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:42.488771 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:42.486945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50397 (local address 127.0.0.1:34477)
0806 03:55:42.487162 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:42.487171 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:42.487205 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:55:42.487326 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:42.487332 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:42.487361 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:42.487508 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:42.488199 (+ 691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:42.488203 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:42.488204 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:42.488208 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:42.488254 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:42.488268 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:42.488321 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:42.488492 (+ 171us) server_negotiation.cc:299] Negotiation successful
0806 03:55:42.488539 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":99,"threads_started":1}
I20260806 03:55:42.489099 11531 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:42.489759 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:42.490820 11531 catalog_manager.cc:1347] Generated new cluster ID: 62eecbbc2f0e44b29a399893e9e54fa3
I20260806 03:55:42.490875 11531 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:42.499468 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:42.498449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49477 (local address 127.0.0.1:34477)
0806 03:55:42.498485 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:42.498486 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:42.498615 (+ 129us) server_negotiation.cc:407] Connection header received
0806 03:55:42.498693 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:42.498694 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:42.498699 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:42.498743 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:42.499271 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:42.499273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:42.499273 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:42.499277 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:42.499292 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:42.499297 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:42.499343 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:42.499390 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:55:42.499403 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:55:42.516305 11531 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:42.516497 11531 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:42.527779 11531 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 91e3d02a77ba46e1ba73524b5f3cfcec: Generated new TSK 0
I20260806 03:55:42.527984 11531 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:42.532233 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 11414
2026-08-06T03:55:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:42 dist-test-slave-bqcl krb5kdc[11565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:42 dist-test-slave-bqcl krb5kdc[11565](info): set up 2 sockets
Aug 06 03:55:42 dist-test-slave-bqcl krb5kdc[11565](info): commencing operation
krb5kdc: starting...
W20260806 03:55:43.666462 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.108s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:43 dist-test-slave-bqcl krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:43.780483 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46759
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46759
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:43.860631 11590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:43.860821 11590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:43.860842 11590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:43.861912 11590 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:43.861946 11590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:43.861959 11590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:43.861970 11590 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:43.861984 11590 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:43.863441 11590 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46759
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46759
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:43.863631 11590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:43.863802 11590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:43.868371 11601 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:43.868361 11602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:43.868446 11590 server_base.cc:1034] running on GCE node
W20260806 03:55:43.868363 11605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:43.868779 11590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:43.869087 11590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:43.870178 11590 hybrid_clock.cc:648] HybridClock initialized: now 1785988543870170 us; error 33 us; skew 500 ppm
Aug 06 03:55:43 dist-test-slave-bqcl krb5kdc[11565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988543, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:43.872483 11590 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:43.872905 11590 webserver.cc:460] Webserver started at http://127.0.0.1:34951/ using document root <none> and password file <none>
I20260806 03:55:43.873124 11590 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:43.873173 11590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:43.873298 11590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:43.874353 11590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "6d0cf67678154862aec40925a7c1dfcb"
format_stamp: "Formatted at 2026-08-06 03:55:43 on dist-test-slave-bqcl"
server_key: "2961ad6fe81977f303ecf2c77b584f0e"
server_key_iv: "31eaf9ceeda1aa6eae47928415aaaece"
server_key_version: "encryptionkey@0"
I20260806 03:55:43.874675 11590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "6d0cf67678154862aec40925a7c1dfcb"
format_stamp: "Formatted at 2026-08-06 03:55:43 on dist-test-slave-bqcl"
server_key: "2961ad6fe81977f303ecf2c77b584f0e"
server_key_iv: "31eaf9ceeda1aa6eae47928415aaaece"
server_key_version: "encryptionkey@0"
I20260806 03:55:43.875926 11590 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:55:43.876582 11621 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:43.876725 11590 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:43.876820 11590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "6d0cf67678154862aec40925a7c1dfcb"
format_stamp: "Formatted at 2026-08-06 03:55:43 on dist-test-slave-bqcl"
server_key: "2961ad6fe81977f303ecf2c77b584f0e"
server_key_iv: "31eaf9ceeda1aa6eae47928415aaaece"
server_key_version: "encryptionkey@0"
I20260806 03:55:43.876879 11590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:43.887727 11590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:43.889283 11590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:43.889412 11590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:43.893255 11590 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46759
I20260806 03:55:43.893314 11701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46759 every 8 connection(s)
I20260806 03:55:43.893597 11590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:43.894447 11702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:43.894520 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 11590
I20260806 03:55:43.894630 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:43.894815 12349 external_mini_cluster.cc:1428] Setting key 034b8745c2335dd929c6d8ed51726524
I20260806 03:55:43.897511 11702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb: Bootstrap starting.
I20260806 03:55:43.898151 11702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:43.898458 11702 log.cc:824] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:43.899109 11702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb: No bootstrap required, opened a new log
Aug 06 03:55:43 dist-test-slave-bqcl krb5kdc[11565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988543, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 03:55:43.901352 11702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0cf67678154862aec40925a7c1dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } }
I20260806 03:55:43.901521 11702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:43.901569 11702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d0cf67678154862aec40925a7c1dfcb, State: Initialized, Role: FOLLOWER
I20260806 03:55:43.901705 11702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0cf67678154862aec40925a7c1dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } }
I20260806 03:55:43.901795 11702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:43.901837 11702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:43.901885 11702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:43.902426 11702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0cf67678154862aec40925a7c1dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } }
I20260806 03:55:43.902515 11702 leader_election.cc:302] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d0cf67678154862aec40925a7c1dfcb; no voters:
I20260806 03:55:43.902675 11702 leader_election.cc:288] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:43.902709 11708 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:43.902822 11708 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [term 1 LEADER]: Becoming Leader. State: Replica: 6d0cf67678154862aec40925a7c1dfcb, State: Running, Role: LEADER
I20260806 03:55:43.902940 11702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:43.902957 11708 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0cf67678154862aec40925a7c1dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } }
I20260806 03:55:43.903285 11708 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d0cf67678154862aec40925a7c1dfcb. Latest consensus state: current_term: 1 leader_uuid: "6d0cf67678154862aec40925a7c1dfcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0cf67678154862aec40925a7c1dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } } }
I20260806 03:55:43.903378 11708 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:43.903297 11710 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d0cf67678154862aec40925a7c1dfcb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d0cf67678154862aec40925a7c1dfcb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46759 } } }
I20260806 03:55:43.903517 11710 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb [sys.catalog]: This master's current role is: LEADER
W20260806 03:55:43.904690 11732 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:55:43.904737 11732 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:55:43.904798 11719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:43.904982 11719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:43.905627 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:43.895349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35525 (local address 127.0.0.1:46759)
0806 03:55:43.895519 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:43.895529 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:43.895559 (+ 30us) server_negotiation.cc:407] Connection header received
0806 03:55:43.896020 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:43.896026 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:43.896074 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:43.896190 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:43.896934 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:43.897837 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:43.898850 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:43.899290 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:43.899379 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:43.901815 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:43.901820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:43.901820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:43.901833 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:43.903746 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:43.904136 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:43.904138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:43.904139 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:43.904240 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:43.904473 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:43.904474 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:43.904474 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:43.904766 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:43.904917 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:43.905293 (+ 376us) server_negotiation.cc:299] Negotiation successful
0806 03:55:43.905431 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":69,"threads_started":1}
I20260806 03:55:43.906487 11719 catalog_manager.cc:1347] Generated new cluster ID: 633dd37fb1b74fbdadfd6e751cc80e67
I20260806 03:55:43.906481 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:43.906533 11719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:43.917059 11705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:43.910412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59533 (local address 127.0.0.1:46759)
0806 03:55:43.910443 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:43.910445 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:43.910558 (+ 113us) server_negotiation.cc:407] Connection header received
0806 03:55:43.910619 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:43.910620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:43.910628 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:43.910690 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:43.911283 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:43.911985 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:43.912947 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:43.913252 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:43.913304 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:43.914581 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:43.914582 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:43.914583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:43.914589 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:43.916019 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:43.916372 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:43.916373 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:43.916373 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:43.916420 (+ 47us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:43.916589 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:43.916590 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:43.916590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:43.916697 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:43.916775 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:43.916909 (+ 134us) server_negotiation.cc:299] Negotiation successful
0806 03:55:43.916975 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:55:43.930738 11719 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:43.931243 11719 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:43.932900 11719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d0cf67678154862aec40925a7c1dfcb: Generated new TSK 0
I20260806 03:55:43.933063 11719 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:43.935671 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 11590
2026-08-06T03:55:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:43 dist-test-slave-bqcl krb5kdc[11750](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:43 dist-test-slave-bqcl krb5kdc[11750](info): set up 2 sockets
Aug 06 03:55:43 dist-test-slave-bqcl krb5kdc[11750](info): commencing operation
krb5kdc: starting...
W20260806 03:55:45.073396 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:45 dist-test-slave-bqcl krb5kdc[11750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988545, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:45.187045 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33717
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39235
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33717
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:45.266813 11768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:45.267011 11768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:45.267032 11768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:45.268138 11768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:45.268188 11768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:45.268201 11768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:45.268213 11768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:45.268224 11768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:45.269657 11768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39235
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33717
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33717
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:45.269874 11768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:45.270085 11768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:45.274732 11778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:45.274734 11782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:45.274768 11768 server_base.cc:1034] running on GCE node
W20260806 03:55:45.274744 11777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:45.275233 11768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:45.275504 11768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:45.276644 11768 hybrid_clock.cc:648] HybridClock initialized: now 1785988545276630 us; error 28 us; skew 500 ppm
Aug 06 03:55:45 dist-test-slave-bqcl krb5kdc[11750](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988545, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:45.282055 11768 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:45.282513 11768 webserver.cc:460] Webserver started at http://127.0.0.1:44403/ using document root <none> and password file <none>
I20260806 03:55:45.282729 11768 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:45.282779 11768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:45.282903 11768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:45.283892 11768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "14b7f79f718547ee8eed7632ae181871"
format_stamp: "Formatted at 2026-08-06 03:55:45 on dist-test-slave-bqcl"
server_key: "a8670742e658234728895e35882f4781"
server_key_iv: "aaf0e8ba539a9488db881017c081a3e0"
server_key_version: "encryptionkey@0"
I20260806 03:55:45.284225 11768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "14b7f79f718547ee8eed7632ae181871"
format_stamp: "Formatted at 2026-08-06 03:55:45 on dist-test-slave-bqcl"
server_key: "a8670742e658234728895e35882f4781"
server_key_iv: "aaf0e8ba539a9488db881017c081a3e0"
server_key_version: "encryptionkey@0"
I20260806 03:55:45.285454 11768 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:45.286078 11795 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:45.286244 11768 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:45.286329 11768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "14b7f79f718547ee8eed7632ae181871"
format_stamp: "Formatted at 2026-08-06 03:55:45 on dist-test-slave-bqcl"
server_key: "a8670742e658234728895e35882f4781"
server_key_iv: "aaf0e8ba539a9488db881017c081a3e0"
server_key_version: "encryptionkey@0"
I20260806 03:55:45.286388 11768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:45.302235 11768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:45.303910 11768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:45.304037 11768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:45.307857 11768 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33717
I20260806 03:55:45.307889 11873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33717 every 8 connection(s)
I20260806 03:55:45.308223 11768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:45.309036 11875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:45.311261 11875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871: Bootstrap starting.
I20260806 03:55:45.311316 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 11768
I20260806 03:55:45.311411 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:45.311574 12349 external_mini_cluster.cc:1428] Setting key 824d2d68cc72096d02a3741fa2056dab
I20260806 03:55:45.311887 11875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:45.312237 11875 log.cc:824] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:45.312954 11875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871: No bootstrap required, opened a new log
I20260806 03:55:45.315217 11875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b7f79f718547ee8eed7632ae181871" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260806 03:55:45.315346 11875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:45.315372 11875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14b7f79f718547ee8eed7632ae181871, State: Initialized, Role: FOLLOWER
I20260806 03:55:45.315500 11875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b7f79f718547ee8eed7632ae181871" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260806 03:55:45.315573 11875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:45.315605 11875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:45.315649 11875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:45.316284 11875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b7f79f718547ee8eed7632ae181871" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260806 03:55:45.316395 11875 leader_election.cc:302] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14b7f79f718547ee8eed7632ae181871; no voters:
I20260806 03:55:45.316555 11875 leader_election.cc:288] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:45.316588 11883 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:45.316769 11883 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [term 1 LEADER]: Becoming Leader. State: Replica: 14b7f79f718547ee8eed7632ae181871, State: Running, Role: LEADER
I20260806 03:55:45.316871 11875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:45.316920 11883 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b7f79f718547ee8eed7632ae181871" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } }
I20260806 03:55:45.317332 11884 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14b7f79f718547ee8eed7632ae181871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b7f79f718547ee8eed7632ae181871" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } } }
I20260806 03:55:45.317422 11884 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:45.317353 11888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14b7f79f718547ee8eed7632ae181871. Latest consensus state: current_term: 1 leader_uuid: "14b7f79f718547ee8eed7632ae181871" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14b7f79f718547ee8eed7632ae181871" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33717 } } }
I20260806 03:55:45.317505 11888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:45.317667 11879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:45.312044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56601 (local address 127.0.0.1:33717)
0806 03:55:45.312251 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:45.312260 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:45.312292 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:55:45.312823 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:45.312833 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:45.312893 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:45.313048 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:45.313233 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:45.314644 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:45.315404 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:45.316007 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:45.316116 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:45.316760 (+ 644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:45.316765 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:45.316765 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:45.316959 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:45.316999 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:45.317014 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:45.317078 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:45.317176 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 03:55:45.317244 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":86,"threads_started":1}
I20260806 03:55:45.318212 11895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:45.318723 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:45.318778 11895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:45.320027 11895 catalog_manager.cc:1347] Generated new cluster ID: 7f143202026e492db5e37ddf8caaa306
I20260806 03:55:45.320078 11895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:45.324198 11895 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:45.324704 11895 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:45.325816 11879 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:45.322411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55721 (local address 127.0.0.1:33717)
0806 03:55:45.322460 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:45.322461 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:45.322623 (+ 162us) server_negotiation.cc:407] Connection header received
0806 03:55:45.322673 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:45.322674 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:45.322683 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:45.322738 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:45.322864 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:45.323570 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:45.324601 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:45.324867 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:45.324930 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:45.325579 (+ 649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:45.325580 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:45.325581 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:45.325585 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:45.325599 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:45.325603 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:45.325658 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:45.325712 (+ 54us) server_negotiation.cc:299] Negotiation successful
0806 03:55:45.325745 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:55:45.332172 11895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14b7f79f718547ee8eed7632ae181871: Generated new TSK 0
I20260806 03:55:45.332321 11895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:45.335705 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 11768
2026-08-06T03:55:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1415 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:45 dist-test-slave-bqcl krb5kdc[11925](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:45 dist-test-slave-bqcl krb5kdc[11925](info): set up 2 sockets
Aug 06 03:55:45 dist-test-slave-bqcl krb5kdc[11925](info): commencing operation
krb5kdc: starting...
W20260806 03:55:46.478423 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:46 dist-test-slave-bqcl krb5kdc[11925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988546, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:46.596390 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36627
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41945
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36627
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:46.677004 11940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:46.677217 11940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:46.677240 11940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:46.678375 11940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:46.678414 11940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:46.678428 11940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:46.678439 11940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:46.678450 11940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:46.679877 11940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41945
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36627
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36627
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11940
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:46.680075 11940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:46.680246 11940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:46.684787 11949 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:46.684778 11953 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:46.684806 11940 server_base.cc:1034] running on GCE node
W20260806 03:55:46.684757 11948 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:46.685221 11940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:46.685526 11940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:46.686667 11940 hybrid_clock.cc:648] HybridClock initialized: now 1785988546686642 us; error 36 us; skew 500 ppm
Aug 06 03:55:46 dist-test-slave-bqcl krb5kdc[11925](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988546, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:46.689067 11940 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:46.689508 11940 webserver.cc:460] Webserver started at http://127.0.0.1:36587/ using document root <none> and password file <none>
I20260806 03:55:46.689723 11940 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:46.689769 11940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:46.689895 11940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:46.691016 11940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "15aa981a9f2d48cbb94c6159668ff41d"
format_stamp: "Formatted at 2026-08-06 03:55:46 on dist-test-slave-bqcl"
server_key: "5af4dc272b1cdf12eeb243e4850c37ed"
server_key_iv: "f500b7f0574a9eba69089345d577e4a0"
server_key_version: "encryptionkey@0"
I20260806 03:55:46.691357 11940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "15aa981a9f2d48cbb94c6159668ff41d"
format_stamp: "Formatted at 2026-08-06 03:55:46 on dist-test-slave-bqcl"
server_key: "5af4dc272b1cdf12eeb243e4850c37ed"
server_key_iv: "f500b7f0574a9eba69089345d577e4a0"
server_key_version: "encryptionkey@0"
I20260806 03:55:46.692663 11940 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:55:46.693322 11964 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:46.693475 11940 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:46.693558 11940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "15aa981a9f2d48cbb94c6159668ff41d"
format_stamp: "Formatted at 2026-08-06 03:55:46 on dist-test-slave-bqcl"
server_key: "5af4dc272b1cdf12eeb243e4850c37ed"
server_key_iv: "f500b7f0574a9eba69089345d577e4a0"
server_key_version: "encryptionkey@0"
I20260806 03:55:46.693627 11940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:46.697157 11940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:46.699635 11940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:46.699759 11940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:46.705088 11940 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36627
I20260806 03:55:46.705571 11940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:46.705920 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 11940
I20260806 03:55:46.706041 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:46.706243 12349 external_mini_cluster.cc:1428] Setting key 70def60d0136f538c49869ceaf261dc7
I20260806 03:55:46.706440 12054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:46.706949 12053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36627 every 8 connection(s)
I20260806 03:55:46.709893 12054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d: Bootstrap starting.
I20260806 03:55:46.710366 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:46.707237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36057 (local address 127.0.0.1:36627)
0806 03:55:46.708330 (+ 1093us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:46.708341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:46.708376 (+ 35us) server_negotiation.cc:407] Connection header received
0806 03:55:46.708951 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:46.708961 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:46.708996 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:46.709141 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:46.709942 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:46.709946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:46.709946 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:46.709951 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:46.709988 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:46.709998 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:46.710056 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:46.710165 (+ 109us) server_negotiation.cc:299] Negotiation successful
0806 03:55:46.710207 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1034,"thread_start_us":85,"threads_started":1}
I20260806 03:55:46.710630 12054 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:46.710942 12054 log.cc:824] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:46.711277 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:46.711684 12054 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d: No bootstrap required, opened a new log
I20260806 03:55:46.713773 12059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:46.712847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41795 (local address 127.0.0.1:36627)
0806 03:55:46.712884 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:46.712887 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:46.713005 (+ 118us) server_negotiation.cc:407] Connection header received
0806 03:55:46.713067 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:46.713068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:46.713080 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:46.713123 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:46.713608 (+ 485us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:46.713609 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:46.713609 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:46.713613 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:46.713623 (+ 10us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:46.713626 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:46.713664 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:46.713711 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:55:46.713720 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:55:46.714161 12054 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa981a9f2d48cbb94c6159668ff41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36627 } }
I20260806 03:55:46.714326 12054 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:46.714357 12054 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15aa981a9f2d48cbb94c6159668ff41d, State: Initialized, Role: FOLLOWER
I20260806 03:55:46.714455 12054 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa981a9f2d48cbb94c6159668ff41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36627 } }
I20260806 03:55:46.714524 12054 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:46.714555 12054 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:46.714587 12054 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:46.715157 12054 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa981a9f2d48cbb94c6159668ff41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36627 } }
I20260806 03:55:46.715246 12054 leader_election.cc:302] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15aa981a9f2d48cbb94c6159668ff41d; no voters:
I20260806 03:55:46.715364 12054 leader_election.cc:288] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:46.715435 12071 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:46.715559 12054 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:46.715607 12071 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [term 1 LEADER]: Becoming Leader. State: Replica: 15aa981a9f2d48cbb94c6159668ff41d, State: Running, Role: LEADER
I20260806 03:55:46.715698 12071 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa981a9f2d48cbb94c6159668ff41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36627 } }
I20260806 03:55:46.716017 12071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15aa981a9f2d48cbb94c6159668ff41d. Latest consensus state: current_term: 1 leader_uuid: "15aa981a9f2d48cbb94c6159668ff41d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa981a9f2d48cbb94c6159668ff41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36627 } } }
I20260806 03:55:46.716092 12071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:46.716053 12072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15aa981a9f2d48cbb94c6159668ff41d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15aa981a9f2d48cbb94c6159668ff41d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36627 } } }
I20260806 03:55:46.716143 12072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:46.716830 12086 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:46.717338 12086 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:46.718680 12086 catalog_manager.cc:1347] Generated new cluster ID: 0da8bb18b2d74750b4d5dca0ca111360
I20260806 03:55:46.718735 12086 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:46.732242 12086 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:46.732400 12086 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:46.745936 12086 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15aa981a9f2d48cbb94c6159668ff41d: Generated new TSK 0
I20260806 03:55:46.746152 12086 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:46.749300 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 11940
2026-08-06T03:55:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:46 dist-test-slave-bqcl krb5kdc[12104](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:46 dist-test-slave-bqcl krb5kdc[12104](info): set up 2 sockets
Aug 06 03:55:46 dist-test-slave-bqcl krb5kdc[12104](info): commencing operation
krb5kdc: starting...
W20260806 03:55:47.885635 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:47 dist-test-slave-bqcl krb5kdc[12104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:48.000552 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35051
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35051
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:48.081278 12121 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:48.081482 12121 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:48.081503 12121 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:48.082633 12121 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:48.082672 12121 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:48.082685 12121 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:48.082696 12121 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:48.082707 12121 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:48.084124 12121 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35051
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35051
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12121
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:48.084312 12121 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:48.084517 12121 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:48.089128 12136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:48.089148 12132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:48.089135 12130 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:48.089191 12121 server_base.cc:1034] running on GCE node
I20260806 03:55:48.089581 12121 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:48.089840 12121 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:48.090974 12121 hybrid_clock.cc:648] HybridClock initialized: now 1785988548090963 us; error 27 us; skew 500 ppm
Aug 06 03:55:48 dist-test-slave-bqcl krb5kdc[12104](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988548, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:48.093331 12121 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:48.093715 12121 webserver.cc:460] Webserver started at http://127.0.0.1:32943/ using document root <none> and password file <none>
I20260806 03:55:48.093906 12121 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:48.093952 12121 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:48.094072 12121 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:48.095074 12121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf"
format_stamp: "Formatted at 2026-08-06 03:55:48 on dist-test-slave-bqcl"
server_key: "29c00abd1af76965af7e17685ef5f40f"
server_key_iv: "a40ba5f7f3d1ff5b35ab4e1f6a3dced1"
server_key_version: "encryptionkey@0"
I20260806 03:55:48.095410 12121 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf"
format_stamp: "Formatted at 2026-08-06 03:55:48 on dist-test-slave-bqcl"
server_key: "29c00abd1af76965af7e17685ef5f40f"
server_key_iv: "a40ba5f7f3d1ff5b35ab4e1f6a3dced1"
server_key_version: "encryptionkey@0"
I20260806 03:55:48.096637 12121 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:55:48.097313 12145 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:48.097465 12121 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:48.097548 12121 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf"
format_stamp: "Formatted at 2026-08-06 03:55:48 on dist-test-slave-bqcl"
server_key: "29c00abd1af76965af7e17685ef5f40f"
server_key_iv: "a40ba5f7f3d1ff5b35ab4e1f6a3dced1"
server_key_version: "encryptionkey@0"
I20260806 03:55:48.097612 12121 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:48.111146 12121 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:48.112986 12121 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:48.113112 12121 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:48.116984 12121 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35051
I20260806 03:55:48.117043 12215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35051 every 8 connection(s)
I20260806 03:55:48.117393 12121 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:48.118357 12216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:48.120594 12216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf: Bootstrap starting.
I20260806 03:55:48.121136 12216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:48.121388 12216 log.cc:824] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:48.122035 12216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf: No bootstrap required, opened a new log
I20260806 03:55:48.123584 12216 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } }
I20260806 03:55:48.123719 12216 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:48.123741 12216 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc581a7c6e2049cb9c5a9c7aad3cbdbf, State: Initialized, Role: FOLLOWER
I20260806 03:55:48.123878 12216 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } }
I20260806 03:55:48.123952 12216 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:48.123986 12216 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:48.124033 12216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:48.124523 12216 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } }
I20260806 03:55:48.124616 12216 leader_election.cc:302] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc581a7c6e2049cb9c5a9c7aad3cbdbf; no voters:
I20260806 03:55:48.124775 12216 leader_election.cc:288] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:48.124818 12221 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:48.124994 12221 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [term 1 LEADER]: Becoming Leader. State: Replica: fc581a7c6e2049cb9c5a9c7aad3cbdbf, State: Running, Role: LEADER
I20260806 03:55:48.125051 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12121
I20260806 03:55:48.125007 12216 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:48.125106 12221 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } }
I20260806 03:55:48.125195 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:48.125391 12349 external_mini_cluster.cc:1428] Setting key 03ea209730dd434f85543d4274dfde25
I20260806 03:55:48.125474 12223 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc581a7c6e2049cb9c5a9c7aad3cbdbf. Latest consensus state: current_term: 1 leader_uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } } }
I20260806 03:55:48.125556 12223 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:48.125604 12222 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc581a7c6e2049cb9c5a9c7aad3cbdbf" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35051 } } }
I20260806 03:55:48.125653 12222 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:48.125780 12227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:48.126497 12227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:48.128116 12227 catalog_manager.cc:1347] Generated new cluster ID: 74b6363f75634a73b2a757328a822471
I20260806 03:55:48.128161 12227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:48.131435 12227 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:48.131916 12227 catalog_manager.cc:1504] Loading token signing keys...
Aug 06 03:55:48 dist-test-slave-bqcl krb5kdc[12104](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988547, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 03:55:48.137152 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:48.125961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:52091 (local address 127.0.0.1:35051)
0806 03:55:48.126200 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:48.126210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:48.126389 (+ 179us) server_negotiation.cc:407] Connection header received
0806 03:55:48.126665 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:48.126674 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:48.126741 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:48.126861 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:48.127700 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:48.129054 (+ 1354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:48.130107 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:48.130764 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:48.130870 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:48.133656 (+ 2786us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:48.133660 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:48.133660 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:48.133672 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:48.135583 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:48.135993 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:48.135996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:48.135996 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:48.136103 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:48.136306 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:48.136307 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:48.136307 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:48.136574 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:48.136750 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:48.136829 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 03:55:48.136929 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":64,"threads_started":1}
I20260806 03:55:48.138108 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:48.139286 12227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc581a7c6e2049cb9c5a9c7aad3cbdbf: Generated new TSK 0
I20260806 03:55:48.139441 12227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:48.145859 12237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:48.139751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:54581 (local address 127.0.0.1:35051)
0806 03:55:48.139788 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:48.139790 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:48.139797 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:55:48.139883 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:48.139884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:48.139892 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:48.139937 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:48.140484 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:48.140990 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:48.142005 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:48.142260 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:48.142306 (+ 46us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:48.143572 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:48.143573 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:48.143574 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:48.143580 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:48.144796 (+ 1216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:48.145188 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:48.145188 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:48.145189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:48.145248 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:48.145444 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:48.145444 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:48.145445 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:48.145552 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:48.145626 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:48.145759 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 03:55:48.145806 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:55:48.147837 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12121
W20260806 03:55:48.161391 12101 connection.cc:535] client connection to 127.0.0.1:35051 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:35051): Connection reset by peer (error 104)
2026-08-06T03:55:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1394 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:48 dist-test-slave-bqcl krb5kdc[12267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:48 dist-test-slave-bqcl krb5kdc[12267](info): set up 2 sockets
Aug 06 03:55:48 dist-test-slave-bqcl krb5kdc[12267](info): commencing operation
krb5kdc: starting...
W20260806 03:55:49.280336 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:49 dist-test-slave-bqcl krb5kdc[12267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988549, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:49.397295 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41161
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38331
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:49.478012 12285 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:49.478264 12285 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:49.478303 12285 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:49.479421 12285 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:49.479461 12285 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:49.479480 12285 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:49.479496 12285 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:49.479511 12285 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:49.480943 12285 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38331
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12285
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:49.481173 12285 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:49.481403 12285 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:49.485939 12290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:49.485970 12295 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:49.486052 12285 server_base.cc:1034] running on GCE node
W20260806 03:55:49.486220 12292 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:49.486478 12285 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:49.486747 12285 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:49.487872 12285 hybrid_clock.cc:648] HybridClock initialized: now 1785988549487857 us; error 32 us; skew 500 ppm
Aug 06 03:55:49 dist-test-slave-bqcl krb5kdc[12267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988549, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:49.490247 12285 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:49.490693 12285 webserver.cc:460] Webserver started at http://127.0.0.1:36975/ using document root <none> and password file <none>
I20260806 03:55:49.490893 12285 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:49.490940 12285 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:49.491066 12285 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:49.492070 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "069821070c1a454aa52041a493e51e3e"
format_stamp: "Formatted at 2026-08-06 03:55:49 on dist-test-slave-bqcl"
server_key: "5f12f90a604e48f631358a20464f83a5"
server_key_iv: "03e92d5a5f08f4c339492b8cb0c0bec5"
server_key_version: "encryptionkey@0"
I20260806 03:55:49.492398 12285 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "069821070c1a454aa52041a493e51e3e"
format_stamp: "Formatted at 2026-08-06 03:55:49 on dist-test-slave-bqcl"
server_key: "5f12f90a604e48f631358a20464f83a5"
server_key_iv: "03e92d5a5f08f4c339492b8cb0c0bec5"
server_key_version: "encryptionkey@0"
I20260806 03:55:49.493603 12285 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:49.494262 12302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:49.494407 12285 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:49.494490 12285 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "069821070c1a454aa52041a493e51e3e"
format_stamp: "Formatted at 2026-08-06 03:55:49 on dist-test-slave-bqcl"
server_key: "5f12f90a604e48f631358a20464f83a5"
server_key_iv: "03e92d5a5f08f4c339492b8cb0c0bec5"
server_key_version: "encryptionkey@0"
I20260806 03:55:49.494557 12285 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:49.502585 12285 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:49.504177 12285 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:49.504297 12285 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:49.508199 12285 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41161
I20260806 03:55:49.508232 12387 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41161 every 8 connection(s)
I20260806 03:55:49.508534 12285 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:49.509334 12388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:49.511556 12388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e: Bootstrap starting.
I20260806 03:55:49.511933 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12285
I20260806 03:55:49.512041 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:49.512207 12388 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:49.512253 12349 external_mini_cluster.cc:1428] Setting key 7538d3204a6462dc1b1fa00a6c65a98f
I20260806 03:55:49.512511 12388 log.cc:824] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:49.513329 12388 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e: No bootstrap required, opened a new log
I20260806 03:55:49.515350 12388 raft_consensus.cc:348] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069821070c1a454aa52041a493e51e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260806 03:55:49.515518 12388 raft_consensus.cc:374] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:49.515548 12388 raft_consensus.cc:729] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 069821070c1a454aa52041a493e51e3e, State: Initialized, Role: FOLLOWER
I20260806 03:55:49.515674 12388 consensus_queue.cc:260] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069821070c1a454aa52041a493e51e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260806 03:55:49.515746 12388 raft_consensus.cc:388] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:49.515787 12388 raft_consensus.cc:482] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:49.515838 12388 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:49.516340 12388 raft_consensus.cc:504] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069821070c1a454aa52041a493e51e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260806 03:55:49.516450 12388 leader_election.cc:302] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 069821070c1a454aa52041a493e51e3e; no voters:
I20260806 03:55:49.516620 12388 leader_election.cc:288] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:49.516639 12396 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:49.516773 12396 raft_consensus.cc:686] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [term 1 LEADER]: Becoming Leader. State: Replica: 069821070c1a454aa52041a493e51e3e, State: Running, Role: LEADER
I20260806 03:55:49.516882 12388 sys_catalog.cc:564] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:49.516925 12396 consensus_queue.cc:237] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069821070c1a454aa52041a493e51e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } }
I20260806 03:55:49.517311 12397 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 069821070c1a454aa52041a493e51e3e. Latest consensus state: current_term: 1 leader_uuid: "069821070c1a454aa52041a493e51e3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069821070c1a454aa52041a493e51e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } } }
I20260806 03:55:49.517381 12397 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:49.517580 12402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:49.517902 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:49.512763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:44819 (local address 127.0.0.1:41161)
0806 03:55:49.512931 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:49.512938 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:49.512961 (+ 23us) server_negotiation.cc:407] Connection header received
0806 03:55:49.513382 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:49.513391 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:49.513453 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:49.513565 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:49.513802 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:49.514726 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:49.515812 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:49.516260 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:49.516336 (+ 76us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:49.517141 (+ 805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:49.517145 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:49.517145 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:49.517334 (+ 189us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:49.517376 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:49.517392 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:49.517474 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:49.517675 (+ 201us) server_negotiation.cc:299] Negotiation successful
0806 03:55:49.517705 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260806 03:55:49.518304 12398 sys_catalog.cc:455] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "069821070c1a454aa52041a493e51e3e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "069821070c1a454aa52041a493e51e3e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41161 } } }
I20260806 03:55:49.518419 12398 sys_catalog.cc:458] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:49.518452 12402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:49.518997 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:49.522501 12402 catalog_manager.cc:1347] Generated new cluster ID: 6c81059955884f4f9383dabc85fb187c
I20260806 03:55:49.522557 12402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:49.522863 12395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:49.520208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:52977 (local address 127.0.0.1:41161)
0806 03:55:49.520241 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:49.520243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:49.520352 (+ 109us) server_negotiation.cc:407] Connection header received
0806 03:55:49.520407 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:49.520408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:49.520417 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:49.520459 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:49.520579 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:49.521038 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:49.521699 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:49.521915 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:49.521982 (+ 67us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:49.522586 (+ 604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:49.522587 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:49.522587 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:49.522591 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:49.522613 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:49.522617 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:49.522668 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:49.522776 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 03:55:49.522789 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:55:49.529312 12402 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:49.529819 12402 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:49.536738 12402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 069821070c1a454aa52041a493e51e3e: Generated new TSK 0
I20260806 03:55:49.536906 12402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:49.541610 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12285
W20260806 03:55:49.568867 12264 connection.cc:535] client connection to 127.0.0.1:41161 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41161): Connection reset by peer (error 104)
2026-08-06T03:55:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:49 dist-test-slave-bqcl krb5kdc[12435](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:49 dist-test-slave-bqcl krb5kdc[12435](info): set up 2 sockets
Aug 06 03:55:49 dist-test-slave-bqcl krb5kdc[12435](info): commencing operation
krb5kdc: starting...
W20260806 03:55:50.705637 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:50 dist-test-slave-bqcl krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988550, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:50.819061 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37537
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:50.898597 12451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:50.898795 12451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:50.898818 12451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:50.899951 12451 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:50.899991 12451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:50.900003 12451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:50.900015 12451 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:50.900032 12451 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:50.901437 12451 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12451
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:50.901625 12451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:50.901811 12451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:50.906414 12460 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:50.906541 12451 server_base.cc:1034] running on GCE node
W20260806 03:55:50.906425 12463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:50.906467 12466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:50.906824 12451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:50.907119 12451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:50.908268 12451 hybrid_clock.cc:648] HybridClock initialized: now 1785988550908248 us; error 30 us; skew 500 ppm
Aug 06 03:55:50 dist-test-slave-bqcl krb5kdc[12435](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988550, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:50.910626 12451 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:50.911026 12451 webserver.cc:460] Webserver started at http://127.0.0.1:40855/ using document root <none> and password file <none>
I20260806 03:55:50.911237 12451 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:50.911280 12451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:50.911401 12451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:50.912370 12451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "a6b1b529f57945b8862452c517108d40"
format_stamp: "Formatted at 2026-08-06 03:55:50 on dist-test-slave-bqcl"
server_key: "4c2c739000eae6f721550978ab791765"
server_key_iv: "9bf066f95e372487820f3c593654f89b"
server_key_version: "encryptionkey@0"
I20260806 03:55:50.912689 12451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "a6b1b529f57945b8862452c517108d40"
format_stamp: "Formatted at 2026-08-06 03:55:50 on dist-test-slave-bqcl"
server_key: "4c2c739000eae6f721550978ab791765"
server_key_iv: "9bf066f95e372487820f3c593654f89b"
server_key_version: "encryptionkey@0"
I20260806 03:55:50.913929 12451 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:55:50.914654 12474 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:50.914795 12451 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:50.914873 12451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "a6b1b529f57945b8862452c517108d40"
format_stamp: "Formatted at 2026-08-06 03:55:50 on dist-test-slave-bqcl"
server_key: "4c2c739000eae6f721550978ab791765"
server_key_iv: "9bf066f95e372487820f3c593654f89b"
server_key_version: "encryptionkey@0"
I20260806 03:55:50.914942 12451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:50.918032 12451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:50.919662 12451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:50.919777 12451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:50.923535 12451 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37537
I20260806 03:55:50.923565 12553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37537 every 8 connection(s)
I20260806 03:55:50.923844 12451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:50.924763 12554 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:50.926899 12554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40: Bootstrap starting.
I20260806 03:55:50.927416 12554 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:50.927662 12554 log.cc:824] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:50.928287 12554 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40: No bootstrap required, opened a new log
I20260806 03:55:50.930227 12554 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b1b529f57945b8862452c517108d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37537 } }
I20260806 03:55:50.930364 12554 raft_consensus.cc:374] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:50.930397 12554 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6b1b529f57945b8862452c517108d40, State: Initialized, Role: FOLLOWER
I20260806 03:55:50.930531 12554 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b1b529f57945b8862452c517108d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37537 } }
I20260806 03:55:50.930601 12554 raft_consensus.cc:388] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:50.930622 12554 raft_consensus.cc:482] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:50.930665 12554 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:50.931169 12554 raft_consensus.cc:504] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b1b529f57945b8862452c517108d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37537 } }
I20260806 03:55:50.931258 12554 leader_election.cc:302] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a6b1b529f57945b8862452c517108d40; no voters:
I20260806 03:55:50.931406 12554 leader_election.cc:288] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:50.931442 12560 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:50.931586 12560 raft_consensus.cc:686] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [term 1 LEADER]: Becoming Leader. State: Replica: a6b1b529f57945b8862452c517108d40, State: Running, Role: LEADER
I20260806 03:55:50.931659 12554 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:50.931676 12560 consensus_queue.cc:237] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b1b529f57945b8862452c517108d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37537 } }
I20260806 03:55:50.932017 12562 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a6b1b529f57945b8862452c517108d40. Latest consensus state: current_term: 1 leader_uuid: "a6b1b529f57945b8862452c517108d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b1b529f57945b8862452c517108d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37537 } } }
I20260806 03:55:50.932129 12561 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a6b1b529f57945b8862452c517108d40" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a6b1b529f57945b8862452c517108d40" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37537 } } }
I20260806 03:55:50.932163 12562 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:50.932221 12561 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:50.932600 12572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:50.933100 12572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:50.933303 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12451
I20260806 03:55:50.933419 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:50.933588 12349 external_mini_cluster.cc:1428] Setting key 660659ba2ac0ccdd0b7f235281533d4f
I20260806 03:55:50.934754 12572 catalog_manager.cc:1347] Generated new cluster ID: db0a06207342488fad991d370ebcdb63
I20260806 03:55:50.934808 12572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:50.939661 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:50.934174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:45531 (local address 127.0.0.1:37537)
0806 03:55:50.934470 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:50.934479 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:50.934511 (+ 32us) server_negotiation.cc:407] Connection header received
0806 03:55:50.934620 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:50.934626 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:50.934666 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:50.938299 (+ 3633us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:50.939226 (+ 927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:50.939229 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:50.939230 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:50.939233 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:50.939268 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:50.939281 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:50.939343 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:50.939424 (+ 81us) server_negotiation.cc:299] Negotiation successful
0806 03:55:50.939466 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":90,"threads_started":1}
I20260806 03:55:50.940573 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:50.943099 12586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:50.942064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:53445 (local address 127.0.0.1:37537)
0806 03:55:50.942163 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:50.942165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:50.942276 (+ 111us) server_negotiation.cc:407] Connection header received
0806 03:55:50.942339 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:50.942339 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:50.942343 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:50.942388 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:50.942916 (+ 528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:50.942916 (+ 0us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:50.942917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:50.942920 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:50.942934 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:50.942938 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:50.942985 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:50.943030 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 03:55:50.943041 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260806 03:55:50.949800 12572 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:50.949954 12572 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:50.955572 12572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a6b1b529f57945b8862452c517108d40: Generated new TSK 0
I20260806 03:55:50.955739 12572 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:50.959959 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12451
2026-08-06T03:55:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:51 dist-test-slave-bqcl krb5kdc[12597](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:51 dist-test-slave-bqcl krb5kdc[12597](info): set up 2 sockets
Aug 06 03:55:51 dist-test-slave-bqcl krb5kdc[12597](info): commencing operation
krb5kdc: starting...
W20260806 03:55:52.099475 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.114s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:52 dist-test-slave-bqcl krb5kdc[12597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:52.214746 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39235
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39235
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:52.294435 12614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:52.294631 12614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:52.294651 12614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:52.295720 12614 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:52.295753 12614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:52.295764 12614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:52.295775 12614 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:52.295792 12614 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:52.297158 12614 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39235
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12614
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:52.297334 12614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:52.297497 12614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:52.301962 12621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:52.301962 12625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:52.302030 12622 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:52.302052 12614 server_base.cc:1034] running on GCE node
I20260806 03:55:52.302433 12614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:52.302706 12614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:52.303838 12614 hybrid_clock.cc:648] HybridClock initialized: now 1785988552303825 us; error 28 us; skew 500 ppm
Aug 06 03:55:52 dist-test-slave-bqcl krb5kdc[12597](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988552, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:52.306142 12614 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:52.306545 12614 webserver.cc:460] Webserver started at http://127.0.0.1:32817/ using document root <none> and password file <none>
I20260806 03:55:52.306741 12614 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:52.306790 12614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:52.306913 12614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:52.307973 12614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "62216399b97143a8a7888ff026b6a434"
format_stamp: "Formatted at 2026-08-06 03:55:52 on dist-test-slave-bqcl"
server_key: "1e1a5fd3147ad1592962650fcd6cc2d2"
server_key_iv: "d5b59ddd9338dcdd1bec089e010e3efd"
server_key_version: "encryptionkey@0"
I20260806 03:55:52.308300 12614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "62216399b97143a8a7888ff026b6a434"
format_stamp: "Formatted at 2026-08-06 03:55:52 on dist-test-slave-bqcl"
server_key: "1e1a5fd3147ad1592962650fcd6cc2d2"
server_key_iv: "d5b59ddd9338dcdd1bec089e010e3efd"
server_key_version: "encryptionkey@0"
I20260806 03:55:52.309506 12614 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:52.310175 12636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:52.310310 12614 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:52.310395 12614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "62216399b97143a8a7888ff026b6a434"
format_stamp: "Formatted at 2026-08-06 03:55:52 on dist-test-slave-bqcl"
server_key: "1e1a5fd3147ad1592962650fcd6cc2d2"
server_key_iv: "d5b59ddd9338dcdd1bec089e010e3efd"
server_key_version: "encryptionkey@0"
I20260806 03:55:52.310462 12614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:52.319585 12614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:52.321198 12614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:52.321326 12614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:52.325135 12614 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39235
I20260806 03:55:52.325173 12709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39235 every 8 connection(s)
I20260806 03:55:52.325495 12614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:52.326351 12710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:52.328470 12710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434: Bootstrap starting.
I20260806 03:55:52.328985 12710 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:52.329178 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12614
I20260806 03:55:52.329276 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:52.329286 12710 log.cc:824] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:52.329445 12349 external_mini_cluster.cc:1428] Setting key 343075f93e50fb7303484f25e746e8f8
I20260806 03:55:52.330056 12710 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434: No bootstrap required, opened a new log
I20260806 03:55:52.332134 12710 raft_consensus.cc:348] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62216399b97143a8a7888ff026b6a434" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39235 } }
I20260806 03:55:52.332264 12710 raft_consensus.cc:374] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:52.332296 12710 raft_consensus.cc:729] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 62216399b97143a8a7888ff026b6a434, State: Initialized, Role: FOLLOWER
I20260806 03:55:52.332419 12710 consensus_queue.cc:260] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62216399b97143a8a7888ff026b6a434" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39235 } }
I20260806 03:55:52.332484 12710 raft_consensus.cc:388] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:52.332504 12710 raft_consensus.cc:482] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:52.332548 12710 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:52.333055 12710 raft_consensus.cc:504] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62216399b97143a8a7888ff026b6a434" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39235 } }
I20260806 03:55:52.333144 12710 leader_election.cc:302] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 62216399b97143a8a7888ff026b6a434; no voters:
I20260806 03:55:52.333299 12710 leader_election.cc:288] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:52.333334 12716 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:52.333493 12716 raft_consensus.cc:686] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [term 1 LEADER]: Becoming Leader. State: Replica: 62216399b97143a8a7888ff026b6a434, State: Running, Role: LEADER
I20260806 03:55:52.333555 12710 sys_catalog.cc:564] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:52.333644 12716 consensus_queue.cc:237] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62216399b97143a8a7888ff026b6a434" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39235 } }
I20260806 03:55:52.334092 12717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "62216399b97143a8a7888ff026b6a434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62216399b97143a8a7888ff026b6a434" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39235 } } }
I20260806 03:55:52.334209 12717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:52.334491 12727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:52.334985 12727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:52.334154 12718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 62216399b97143a8a7888ff026b6a434. Latest consensus state: current_term: 1 leader_uuid: "62216399b97143a8a7888ff026b6a434" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "62216399b97143a8a7888ff026b6a434" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39235 } } }
I20260806 03:55:52.335456 12718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434 [sys.catalog]: This master's current role is: LEADER
Aug 06 03:55:52 dist-test-slave-bqcl krb5kdc[12597](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988552, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260806 03:55:52.338204 12727 catalog_manager.cc:1347] Generated new cluster ID: 4defc85b436d46d9871cefd3cef99c77
I20260806 03:55:52.338258 12727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:52.339782 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:52.329907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:58825 (local address 127.0.0.1:39235)
0806 03:55:52.330083 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:52.330093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:52.330145 (+ 52us) server_negotiation.cc:407] Connection header received
0806 03:55:52.330571 (+ 426us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:52.330577 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:52.330623 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:52.330746 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:52.331350 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:52.332208 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:52.333023 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:52.333554 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:52.333682 (+ 128us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:52.336561 (+ 2879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:52.336565 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:52.336566 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:52.336577 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:52.338344 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:52.338719 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:52.338721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:52.338722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:52.338825 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:52.338989 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:52.338989 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:52.338990 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:52.339257 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:52.339388 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:52.339498 (+ 110us) server_negotiation.cc:299] Negotiation successful
0806 03:55:52.339591 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122,"thread_start_us":79,"threads_started":1}
I20260806 03:55:52.340610 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:52.347452 12715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:52.341849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:40345 (local address 127.0.0.1:39235)
0806 03:55:52.341885 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:52.341887 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:52.342007 (+ 120us) server_negotiation.cc:407] Connection header received
0806 03:55:52.342029 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:52.342030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:52.342040 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:52.342095 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:52.342570 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:52.343032 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:52.343691 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:52.343912 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:52.343964 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:52.345010 (+ 1046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:52.345012 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:52.345012 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:52.345019 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:52.346418 (+ 1399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:52.346755 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:52.346756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:52.346756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:52.346812 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:52.346999 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:52.347000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:52.347000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:52.347109 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:52.347213 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:52.347316 (+ 103us) server_negotiation.cc:299] Negotiation successful
0806 03:55:52.347373 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:55:52.350589 12727 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:52.351137 12727 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:52.355271 12727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 62216399b97143a8a7888ff026b6a434: Generated new TSK 0
I20260806 03:55:52.355435 12727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:52.357120 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12614
W20260806 03:55:52.365701 12595 connection.cc:535] client connection to 127.0.0.1:39235 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39235): Connection reset by peer (error 104)
2026-08-06T03:55:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1393 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:52 dist-test-slave-bqcl krb5kdc[12752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:52 dist-test-slave-bqcl krb5kdc[12752](info): set up 2 sockets
Aug 06 03:55:52 dist-test-slave-bqcl krb5kdc[12752](info): commencing operation
krb5kdc: starting...
W20260806 03:55:53.481529 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.102s user 0.004s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:53 dist-test-slave-bqcl krb5kdc[12752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988553, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:53.598435 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32853
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:32853
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:53.679023 12770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:53.679214 12770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:53.679242 12770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:53.680327 12770 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:53.680366 12770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:53.680385 12770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:53.680401 12770 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:53.680418 12770 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:53.681846 12770 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:32853
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:32853
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:53.682085 12770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:53.682307 12770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:53.686803 12777 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:53.686829 12770 server_base.cc:1034] running on GCE node
W20260806 03:55:53.686803 12781 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:53.686846 12778 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:53.687296 12770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:53.687577 12770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:53.688714 12770 hybrid_clock.cc:648] HybridClock initialized: now 1785988553688702 us; error 29 us; skew 500 ppm
Aug 06 03:55:53 dist-test-slave-bqcl krb5kdc[12752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988553, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:53.691123 12770 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:53.691511 12770 webserver.cc:460] Webserver started at http://127.0.0.1:36871/ using document root <none> and password file <none>
I20260806 03:55:53.691699 12770 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:53.691746 12770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:53.691871 12770 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:53.692865 12770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "2561a0dac8034c0da1fea3cb4416db9c"
format_stamp: "Formatted at 2026-08-06 03:55:53 on dist-test-slave-bqcl"
server_key: "bacd82d62b3f63c838c1fdd3ae4b6f94"
server_key_iv: "4ed198ba852fe6cff28cc0d8e8caedf4"
server_key_version: "encryptionkey@0"
I20260806 03:55:53.693184 12770 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "2561a0dac8034c0da1fea3cb4416db9c"
format_stamp: "Formatted at 2026-08-06 03:55:53 on dist-test-slave-bqcl"
server_key: "bacd82d62b3f63c838c1fdd3ae4b6f94"
server_key_iv: "4ed198ba852fe6cff28cc0d8e8caedf4"
server_key_version: "encryptionkey@0"
I20260806 03:55:53.694406 12770 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:55:53.695036 12794 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:53.695184 12770 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:53.695276 12770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "2561a0dac8034c0da1fea3cb4416db9c"
format_stamp: "Formatted at 2026-08-06 03:55:53 on dist-test-slave-bqcl"
server_key: "bacd82d62b3f63c838c1fdd3ae4b6f94"
server_key_iv: "4ed198ba852fe6cff28cc0d8e8caedf4"
server_key_version: "encryptionkey@0"
I20260806 03:55:53.695334 12770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:53.714555 12770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:53.716252 12770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:53.716382 12770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:53.720085 12770 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:32853
I20260806 03:55:53.720134 12880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:32853 every 8 connection(s)
I20260806 03:55:53.720440 12770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:53.721238 12882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:53.722919 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12770
I20260806 03:55:53.723011 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:53.723174 12349 external_mini_cluster.cc:1428] Setting key 90e7a8fc011549e212ebd7f9846145be
I20260806 03:55:53.723618 12882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c: Bootstrap starting.
I20260806 03:55:53.724216 12882 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:53.724509 12882 log.cc:824] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:53.725178 12882 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c: No bootstrap required, opened a new log
I20260806 03:55:53.727383 12882 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2561a0dac8034c0da1fea3cb4416db9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } }
I20260806 03:55:53.727528 12882 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:53.727562 12882 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2561a0dac8034c0da1fea3cb4416db9c, State: Initialized, Role: FOLLOWER
I20260806 03:55:53.727699 12882 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2561a0dac8034c0da1fea3cb4416db9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } }
I20260806 03:55:53.727766 12882 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:53.727787 12882 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:53.727834 12882 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:53.728348 12882 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2561a0dac8034c0da1fea3cb4416db9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } }
I20260806 03:55:53.728471 12882 leader_election.cc:302] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2561a0dac8034c0da1fea3cb4416db9c; no voters:
I20260806 03:55:53.728475 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:53.723639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:37885 (local address 127.0.0.1:32853)
0806 03:55:53.723810 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:53.723817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:53.723840 (+ 23us) server_negotiation.cc:407] Connection header received
0806 03:55:53.724262 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:53.724275 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:53.724342 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:53.724458 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:53.724684 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:53.725599 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:53.726692 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:53.727071 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:53.727148 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:53.727856 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:53.727860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:53.727860 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:53.728023 (+ 163us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:53.728053 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:53.728072 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:53.728141 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:53.728254 (+ 113us) server_negotiation.cc:299] Negotiation successful
0806 03:55:53.728281 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":73,"threads_started":1}
I20260806 03:55:53.728659 12882 leader_election.cc:288] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:53.728705 12891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:53.728858 12891 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [term 1 LEADER]: Becoming Leader. State: Replica: 2561a0dac8034c0da1fea3cb4416db9c, State: Running, Role: LEADER
I20260806 03:55:53.728950 12891 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2561a0dac8034c0da1fea3cb4416db9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } }
I20260806 03:55:53.729089 12882 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:53.729342 12892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2561a0dac8034c0da1fea3cb4416db9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2561a0dac8034c0da1fea3cb4416db9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } } }
I20260806 03:55:53.729441 12892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:53.729501 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:53.729342 12893 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2561a0dac8034c0da1fea3cb4416db9c. Latest consensus state: current_term: 1 leader_uuid: "2561a0dac8034c0da1fea3cb4416db9c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2561a0dac8034c0da1fea3cb4416db9c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 32853 } } }
I20260806 03:55:53.730254 12893 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:53.730427 12896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:53.731033 12896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:53.732515 12896 catalog_manager.cc:1347] Generated new cluster ID: b2e696dd1b914d51811c01457314308c
I20260806 03:55:53.732565 12896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:53.734153 12889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:53.730950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:40587 (local address 127.0.0.1:32853)
0806 03:55:53.730983 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:53.730985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:53.731139 (+ 154us) server_negotiation.cc:407] Connection header received
0806 03:55:53.731236 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:53.731237 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:53.731246 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:53.731303 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:53.731444 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:53.731970 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:53.732932 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:53.733228 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:53.733282 (+ 54us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:53.733907 (+ 625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:53.733908 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:53.733909 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:53.733913 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:53.733926 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:53.733930 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:53.733978 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:53.734055 (+ 77us) server_negotiation.cc:299] Negotiation successful
0806 03:55:53.734064 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:55:53.745030 12896 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:53.745494 12896 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:53.748168 12896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2561a0dac8034c0da1fea3cb4416db9c: Generated new TSK 0
I20260806 03:55:53.748338 12896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:53.753093 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12770
W20260806 03:55:53.766667 12750 connection.cc:535] client connection to 127.0.0.1:32853 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:32853): Connection reset by peer (error 104)
2026-08-06T03:55:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:53 dist-test-slave-bqcl krb5kdc[12928](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:53 dist-test-slave-bqcl krb5kdc[12928](info): set up 2 sockets
Aug 06 03:55:53 dist-test-slave-bqcl krb5kdc[12928](info): commencing operation
krb5kdc: starting...
W20260806 03:55:54.893352 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:54 dist-test-slave-bqcl krb5kdc[12928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988554, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260806 03:55:55.009179 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37561
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:55.088918 12951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:55.089118 12951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:55.089148 12951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:55.090281 12951 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:55.090322 12951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:55.090340 12951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:55.090356 12951 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:55.090373 12951 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:55.091827 12951 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37561
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:55.092061 12951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:55.092258 12951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:55.096935 12977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:55.096951 12963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:55.096952 12967 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:55.097208 12951 server_base.cc:1034] running on GCE node
I20260806 03:55:55.097451 12951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:55.097702 12951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:55.098857 12951 hybrid_clock.cc:648] HybridClock initialized: now 1785988555098841 us; error 28 us; skew 500 ppm
Aug 06 03:55:55 dist-test-slave-bqcl krb5kdc[12928](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988555, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:55.101263 12951 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260806 03:55:55.101724 12951 webserver.cc:460] Webserver started at http://127.0.0.1:39337/ using document root <none> and password file <none>
I20260806 03:55:55.101936 12951 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:55.101984 12951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:55.102111 12951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:55.103166 12951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "877cfca5c18c45a09283a049b4c1f805"
format_stamp: "Formatted at 2026-08-06 03:55:55 on dist-test-slave-bqcl"
server_key: "879f7d2cbbb3239b93192934c3c0e5b1"
server_key_iv: "f989ce1f98864ad61a144ecfe2a99acb"
server_key_version: "encryptionkey@0"
I20260806 03:55:55.103498 12951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "877cfca5c18c45a09283a049b4c1f805"
format_stamp: "Formatted at 2026-08-06 03:55:55 on dist-test-slave-bqcl"
server_key: "879f7d2cbbb3239b93192934c3c0e5b1"
server_key_iv: "f989ce1f98864ad61a144ecfe2a99acb"
server_key_version: "encryptionkey@0"
I20260806 03:55:55.104696 12951 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:55:55.105366 12989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:55.105535 12951 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:55.105636 12951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "877cfca5c18c45a09283a049b4c1f805"
format_stamp: "Formatted at 2026-08-06 03:55:55 on dist-test-slave-bqcl"
server_key: "879f7d2cbbb3239b93192934c3c0e5b1"
server_key_iv: "f989ce1f98864ad61a144ecfe2a99acb"
server_key_version: "encryptionkey@0"
I20260806 03:55:55.105697 12951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:55.108978 12951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:55.110719 12951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:55.110841 12951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:55.114579 12951 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37561
I20260806 03:55:55.114609 13071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37561 every 8 connection(s)
I20260806 03:55:55.114923 12951 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:55.115835 13072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:55.118018 13072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805: Bootstrap starting.
I20260806 03:55:55.118561 13072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:55.118801 13072 log.cc:824] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:55.119395 13072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805: No bootstrap required, opened a new log
I20260806 03:55:55.120874 13072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877cfca5c18c45a09283a049b4c1f805" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } }
I20260806 03:55:55.121001 13072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:55.121029 13072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 877cfca5c18c45a09283a049b4c1f805, State: Initialized, Role: FOLLOWER
I20260806 03:55:55.121136 13072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877cfca5c18c45a09283a049b4c1f805" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } }
I20260806 03:55:55.121212 13072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:55.121240 13072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:55.121272 13072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:55.121737 13072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877cfca5c18c45a09283a049b4c1f805" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } }
I20260806 03:55:55.121819 13072 leader_election.cc:302] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 877cfca5c18c45a09283a049b4c1f805; no voters:
I20260806 03:55:55.121940 13072 leader_election.cc:288] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:55.121989 13077 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:55.122181 13072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:55.122200 13077 raft_consensus.cc:686] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [term 1 LEADER]: Becoming Leader. State: Replica: 877cfca5c18c45a09283a049b4c1f805, State: Running, Role: LEADER
I20260806 03:55:55.122367 13077 consensus_queue.cc:237] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877cfca5c18c45a09283a049b4c1f805" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } }
I20260806 03:55:55.122946 13078 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "877cfca5c18c45a09283a049b4c1f805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877cfca5c18c45a09283a049b4c1f805" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } } }
I20260806 03:55:55.122941 13079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 877cfca5c18c45a09283a049b4c1f805. Latest consensus state: current_term: 1 leader_uuid: "877cfca5c18c45a09283a049b4c1f805" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "877cfca5c18c45a09283a049b4c1f805" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37561 } } }
I20260806 03:55:55.123035 13078 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:55.123052 13079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:55.123247 13088 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:55.123600 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 12951
I20260806 03:55:55.123713 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:55.123929 12349 external_mini_cluster.cc:1428] Setting key adb55706919909b1b933031ee9eacf9b
I20260806 03:55:55.124116 13088 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:55.125698 13088 catalog_manager.cc:1347] Generated new cluster ID: 651a15a2069d410dbf9665dfd620725d
I20260806 03:55:55.125780 13088 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:55.126199 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:55.124371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:43467 (local address 127.0.0.1:37561)
0806 03:55:55.124532 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:55.124541 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:55.124637 (+ 96us) server_negotiation.cc:407] Connection header received
0806 03:55:55.124786 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:55.124793 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:55.124821 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:55.124952 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:55.125617 (+ 665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:55.125620 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:55.125621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:55.125625 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:55.125659 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:55.125673 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:55.125738 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:55.125962 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 03:55:55.126004 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260806 03:55:55.127063 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:55.135746 13088 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:55.135886 13088 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:55.136387 13110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:55.128428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.178:45587 (local address 127.0.0.1:37561)
0806 03:55:55.128457 (+ 29us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:55.128458 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:55.135457 (+ 6999us) server_negotiation.cc:407] Connection header received
0806 03:55:55.135516 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:55.135517 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:55.135523 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:55.135552 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:55.136174 (+ 622us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:55.136175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:55.136175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:55.136179 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:55.136203 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:55.136207 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:55.136267 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:55.136312 (+ 45us) server_negotiation.cc:299] Negotiation successful
0806 03:55:55.136324 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 03:55:55.138680 13088 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 877cfca5c18c45a09283a049b4c1f805: Generated new TSK 0
I20260806 03:55:55.138842 13088 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:55.143270 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 12951
2026-08-06T03:55:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:55 dist-test-slave-bqcl krb5kdc[13124](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:55 dist-test-slave-bqcl krb5kdc[13124](info): set up 2 sockets
Aug 06 03:55:55 dist-test-slave-bqcl krb5kdc[13124](info): commencing operation
krb5kdc: starting...
W20260806 03:55:56.278896 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:56 dist-test-slave-bqcl krb5kdc[13124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:55:56.392081 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:46153
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:41071
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:46153
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:56.471164 13156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:56.471381 13156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:56.471405 13156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:56.472532 13156 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:56.472572 13156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:56.472586 13156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:56.472597 13156 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:56.472611 13156 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:56.474095 13156 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:41071
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:46153
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:46153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:56.474380 13156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:56.474694 13156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:56.480363 13163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:56.480347 13165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:56.480338 13162 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:56.480561 13156 server_base.cc:1034] running on GCE node
I20260806 03:55:56.480813 13156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:56.481091 13156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:56.482213 13156 hybrid_clock.cc:648] HybridClock initialized: now 1785988556482203 us; error 28 us; skew 500 ppm
Aug 06 03:55:56 dist-test-slave-bqcl krb5kdc[13124](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988556, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:56.487273 13156 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:55:56.487699 13156 webserver.cc:460] Webserver started at http://127.12.15.126:46679/ using document root <none> and password file <none>
I20260806 03:55:56.487897 13156 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:56.487946 13156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:56.488065 13156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:56.489065 13156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "c8b59949454148559b025b1e15c71b20"
format_stamp: "Formatted at 2026-08-06 03:55:56 on dist-test-slave-bqcl"
server_key: "26f5d4f5c51828856f6d220578df7ae4"
server_key_iv: "a0ffb2552b3a426a28b20f21b2f7faa1"
server_key_version: "encryptionkey@0"
I20260806 03:55:56.489387 13156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "c8b59949454148559b025b1e15c71b20"
format_stamp: "Formatted at 2026-08-06 03:55:56 on dist-test-slave-bqcl"
server_key: "26f5d4f5c51828856f6d220578df7ae4"
server_key_iv: "a0ffb2552b3a426a28b20f21b2f7faa1"
server_key_version: "encryptionkey@0"
I20260806 03:55:56.490644 13156 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:56.491295 13177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:56.491446 13156 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:55:56.491528 13156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "c8b59949454148559b025b1e15c71b20"
format_stamp: "Formatted at 2026-08-06 03:55:56 on dist-test-slave-bqcl"
server_key: "26f5d4f5c51828856f6d220578df7ae4"
server_key_iv: "a0ffb2552b3a426a28b20f21b2f7faa1"
server_key_version: "encryptionkey@0"
I20260806 03:55:56.491587 13156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:56.515457 13156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:56.517163 13156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:56.517313 13156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:56.521080 13156 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:46153
I20260806 03:55:56.521126 13254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:46153 every 8 connection(s)
I20260806 03:55:56.521428 13156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:56.522269 13256 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:56.524489 13256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20: Bootstrap starting.
I20260806 03:55:56.525041 13256 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:56.525312 13256 log.cc:824] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:56.525938 13256 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20: No bootstrap required, opened a new log
I20260806 03:55:56.526371 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13156
I20260806 03:55:56.526487 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:56.526659 12349 external_mini_cluster.cc:1428] Setting key 0cdffedfef3202af4547082f52f550ce
I20260806 03:55:56.527936 13256 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b59949454148559b025b1e15c71b20" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46153 } }
I20260806 03:55:56.528083 13256 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:56.528167 13256 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8b59949454148559b025b1e15c71b20, State: Initialized, Role: FOLLOWER
I20260806 03:55:56.528312 13256 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b59949454148559b025b1e15c71b20" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46153 } }
I20260806 03:55:56.528395 13256 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:56.528435 13256 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:56.528482 13256 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:56.529381 13256 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b59949454148559b025b1e15c71b20" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46153 } }
I20260806 03:55:56.529474 13256 leader_election.cc:302] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8b59949454148559b025b1e15c71b20; no voters:
I20260806 03:55:56.529630 13256 leader_election.cc:288] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:56.529661 13265 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:56.529814 13265 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [term 1 LEADER]: Becoming Leader. State: Replica: c8b59949454148559b025b1e15c71b20, State: Running, Role: LEADER
I20260806 03:55:56.529891 13256 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:56.530241 13265 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b59949454148559b025b1e15c71b20" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46153 } }
I20260806 03:55:56.530747 13270 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8b59949454148559b025b1e15c71b20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b59949454148559b025b1e15c71b20" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46153 } } }
I20260806 03:55:56.530834 13270 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:56.531031 13266 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8b59949454148559b025b1e15c71b20. Latest consensus state: current_term: 1 leader_uuid: "c8b59949454148559b025b1e15c71b20" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8b59949454148559b025b1e15c71b20" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 46153 } } }
I20260806 03:55:56.531332 13266 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:56.531526 13279 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:56.532084 13279 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:56.533502 13279 catalog_manager.cc:1347] Generated new cluster ID: 7ab197b9db1d4d5ea27bb9f7b377867a
I20260806 03:55:56.533553 13279 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:55:56 dist-test-slave-bqcl krb5kdc[13124](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988556, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:55:56.538084 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:56.527170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60613 (local address 127.12.15.126:46153)
0806 03:55:56.527340 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:56.527348 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:56.527372 (+ 24us) server_negotiation.cc:407] Connection header received
0806 03:55:56.527903 (+ 531us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:56.527909 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:56.527950 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:56.528054 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:56.528820 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:56.529819 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:56.530849 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:56.531452 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:56.531558 (+ 106us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:56.534380 (+ 2822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:56.534384 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:56.534384 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:56.534395 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:56.536271 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:56.536681 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:56.536684 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:56.536684 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:56.536815 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:56.537042 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:56.537043 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:56.537043 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:56.537353 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:56.537529 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:56.537771 (+ 242us) server_negotiation.cc:299] Negotiation successful
0806 03:55:56.537884 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":68,"threads_started":1}
I20260806 03:55:56.538358 13279 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:56.538882 13279 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:56.539055 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:56.544795 13279 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8b59949454148559b025b1e15c71b20: Generated new TSK 0
I20260806 03:55:56.544946 13279 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:56.550827 13264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:56.544726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37631 (local address 127.12.15.126:46153)
0806 03:55:56.544793 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:56.544795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:56.544914 (+ 119us) server_negotiation.cc:407] Connection header received
0806 03:55:56.545003 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:56.545004 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:56.545013 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:56.545071 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:56.545671 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:56.546203 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:56.547212 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:56.547389 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:56.547430 (+ 41us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:56.548629 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:56.548630 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:56.548630 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:55:56.548636 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:56.549789 (+ 1153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:56.550210 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:56.550210 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:56.550211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:56.550255 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:55:56.550448 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:55:56.550448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:55:56.550449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:55:56.550530 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:56.550596 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:56.550732 (+ 136us) server_negotiation.cc:299] Negotiation successful
0806 03:55:56.550774 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:55:56.552716 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13156
W20260806 03:55:56.568097 13121 connection.cc:535] client connection to 127.12.15.126:46153 recv error: Network error: failed to read from TLS socket (remote: 127.12.15.126:46153): Connection reset by peer (error 104)
2026-08-06T03:55:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:56 dist-test-slave-bqcl krb5kdc[13306](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:56 dist-test-slave-bqcl krb5kdc[13306](info): set up 2 sockets
Aug 06 03:55:56 dist-test-slave-bqcl krb5kdc[13306](info): commencing operation
krb5kdc: starting...
W20260806 03:55:57.680227 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:57 dist-test-slave-bqcl krb5kdc[13306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988557, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:55:57.794800 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37651
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:34599
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:37651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:57.874377 13330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:57.874570 13330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:57.874603 13330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:57.875705 13330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:57.875746 13330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:57.875763 13330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:57.875778 13330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:57.875793 13330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:57.877225 13330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:34599
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:37651
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:37651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:57.877460 13330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:57.877656 13330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:57.882071 13341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:57.882082 13340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:57.882086 13345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:57.882181 13330 server_base.cc:1034] running on GCE node
I20260806 03:55:57.882546 13330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:57.882807 13330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:57.883941 13330 hybrid_clock.cc:648] HybridClock initialized: now 1785988557883927 us; error 30 us; skew 500 ppm
Aug 06 03:55:57 dist-test-slave-bqcl krb5kdc[13306](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988557, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:57.886256 13330 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:55:57.886658 13330 webserver.cc:460] Webserver started at http://127.12.15.126:43885/ using document root <none> and password file <none>
I20260806 03:55:57.886852 13330 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:57.886894 13330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:57.887017 13330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:57.887977 13330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "42499a1bb6574f51aabea95c578de3d7"
format_stamp: "Formatted at 2026-08-06 03:55:57 on dist-test-slave-bqcl"
server_key: "95bd6df939734204db3ae295b5388703"
server_key_iv: "7f77f9af2dd7351a1bb25565636f4fb5"
server_key_version: "encryptionkey@0"
I20260806 03:55:57.888299 13330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "42499a1bb6574f51aabea95c578de3d7"
format_stamp: "Formatted at 2026-08-06 03:55:57 on dist-test-slave-bqcl"
server_key: "95bd6df939734204db3ae295b5388703"
server_key_iv: "7f77f9af2dd7351a1bb25565636f4fb5"
server_key_version: "encryptionkey@0"
I20260806 03:55:57.889454 13330 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:55:57.890055 13357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:57.890216 13330 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:57.890300 13330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "42499a1bb6574f51aabea95c578de3d7"
format_stamp: "Formatted at 2026-08-06 03:55:57 on dist-test-slave-bqcl"
server_key: "95bd6df939734204db3ae295b5388703"
server_key_iv: "7f77f9af2dd7351a1bb25565636f4fb5"
server_key_version: "encryptionkey@0"
I20260806 03:55:57.890363 13330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:57.901913 13330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:57.903512 13330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:57.903638 13330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:57.907495 13330 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:37651
I20260806 03:55:57.907531 13442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:37651 every 8 connection(s)
I20260806 03:55:57.907825 13330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:57.908632 13443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:57.908861 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13330
I20260806 03:55:57.908975 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:57.909185 12349 external_mini_cluster.cc:1428] Setting key bf9747d31359682ef110c8bf9f12ad29
I20260806 03:55:57.911252 13443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7: Bootstrap starting.
I20260806 03:55:57.911851 13443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:57.912151 13443 log.cc:824] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:57.912839 13443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7: No bootstrap required, opened a new log
I20260806 03:55:57.914327 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:57.909660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37355 (local address 127.12.15.126:37651)
0806 03:55:57.909829 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:57.909838 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:57.909861 (+ 23us) server_negotiation.cc:407] Connection header received
0806 03:55:57.910286 (+ 425us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:57.910293 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:57.910340 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:57.910449 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:57.910674 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:57.911578 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:57.912608 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:57.913031 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:57.913108 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:57.913734 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:57.913738 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:57.913738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:57.913883 (+ 145us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:57.913912 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:57.913922 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:57.913983 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:57.914084 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 03:55:57.914104 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":70,"threads_started":1}
I20260806 03:55:57.914955 13443 raft_consensus.cc:348] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42499a1bb6574f51aabea95c578de3d7" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37651 } }
I20260806 03:55:57.915083 13443 raft_consensus.cc:374] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:57.915117 13443 raft_consensus.cc:729] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 42499a1bb6574f51aabea95c578de3d7, State: Initialized, Role: FOLLOWER
I20260806 03:55:57.915189 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:57.915223 13443 consensus_queue.cc:260] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42499a1bb6574f51aabea95c578de3d7" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37651 } }
I20260806 03:55:57.915293 13443 raft_consensus.cc:388] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:57.915329 13443 raft_consensus.cc:482] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:57.915374 13443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:57.915854 13443 raft_consensus.cc:504] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42499a1bb6574f51aabea95c578de3d7" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37651 } }
I20260806 03:55:57.915951 13443 leader_election.cc:302] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 42499a1bb6574f51aabea95c578de3d7; no voters:
I20260806 03:55:57.916106 13443 leader_election.cc:288] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:57.916134 13454 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:57.916308 13454 raft_consensus.cc:686] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [term 1 LEADER]: Becoming Leader. State: Replica: 42499a1bb6574f51aabea95c578de3d7, State: Running, Role: LEADER
I20260806 03:55:57.916352 13443 sys_catalog.cc:564] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:57.916538 13454 consensus_queue.cc:237] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42499a1bb6574f51aabea95c578de3d7" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37651 } }
I20260806 03:55:57.917719 13459 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "42499a1bb6574f51aabea95c578de3d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42499a1bb6574f51aabea95c578de3d7" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37651 } } }
I20260806 03:55:57.917802 13461 sys_catalog.cc:455] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 42499a1bb6574f51aabea95c578de3d7. Latest consensus state: current_term: 1 leader_uuid: "42499a1bb6574f51aabea95c578de3d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "42499a1bb6574f51aabea95c578de3d7" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 37651 } } }
I20260806 03:55:57.917806 13459 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:57.918094 13461 sys_catalog.cc:458] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7 [sys.catalog]: This master's current role is: LEADER
W20260806 03:55:57.918215 13480 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:55:57.918269 13480 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:55:57.918366 13482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:57.918519 13482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:57.919836 13449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:57.916457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50181 (local address 127.12.15.126:37651)
0806 03:55:57.916491 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:57.916493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:57.916642 (+ 149us) server_negotiation.cc:407] Connection header received
0806 03:55:57.916736 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:57.916737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:57.916746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:57.916800 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:57.916932 (+ 132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:57.917572 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:57.918547 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:55:57.918774 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:55:57.918826 (+ 52us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:55:57.919462 (+ 636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:57.919464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:57.919464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:57.919468 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:57.919480 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:57.919484 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:57.919535 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:57.919756 (+ 221us) server_negotiation.cc:299] Negotiation successful
0806 03:55:57.919765 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260806 03:55:57.919988 13482 catalog_manager.cc:1347] Generated new cluster ID: 7913dc1ada7e40c48c81fcb0440d42a0
I20260806 03:55:57.920033 13482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:57.927245 13482 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:57.927938 13482 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:57.932655 13482 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 42499a1bb6574f51aabea95c578de3d7: Generated new TSK 0
I20260806 03:55:57.932819 13482 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:57.940676 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13330
W20260806 03:55:57.958539 13303 connection.cc:535] client connection to 127.12.15.126:37651 recv error: Network error: failed to read from TLS socket (remote: 127.12.15.126:37651): Connection reset by peer (error 104)
2026-08-06T03:55:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:58 dist-test-slave-bqcl krb5kdc[13491](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:58 dist-test-slave-bqcl krb5kdc[13491](info): set up 2 sockets
Aug 06 03:55:58 dist-test-slave-bqcl krb5kdc[13491](info): commencing operation
krb5kdc: starting...
W20260806 03:55:59.083272 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.111s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:55:59 dist-test-slave-bqcl krb5kdc[13491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988559, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:55:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:55:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:55:59.200448 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:45341
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:35107
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:45341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:55:59.280700 13509 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:55:59.280917 13509 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:55:59.280947 13509 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:55:59.282056 13509 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:55:59.282095 13509 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:55:59.282114 13509 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:55:59.282150 13509 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:55:59.282166 13509 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:55:59.283648 13509 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:35107
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:45341
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:45341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13509
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:55:59.283906 13509 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:55:59.284109 13509 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:55:59.288704 13519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:59.288867 13509 server_base.cc:1034] running on GCE node
W20260806 03:55:59.288715 13516 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:55:59.288949 13514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:55:59.289268 13509 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:55:59.289541 13509 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:55:59.290678 13509 hybrid_clock.cc:648] HybridClock initialized: now 1785988559290663 us; error 30 us; skew 500 ppm
Aug 06 03:55:59 dist-test-slave-bqcl krb5kdc[13491](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988559, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:55:59.293005 13509 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:55:59.293413 13509 webserver.cc:460] Webserver started at http://127.12.15.126:42867/ using document root <none> and password file <none>
I20260806 03:55:59.293613 13509 fs_manager.cc:359] Metadata directory not provided
I20260806 03:55:59.293660 13509 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:55:59.293787 13509 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:55:59.294863 13509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b"
format_stamp: "Formatted at 2026-08-06 03:55:59 on dist-test-slave-bqcl"
server_key: "f52347500d1e8a4a20b416a5e12daa5c"
server_key_iv: "f6fd730d53223bebef1af07d5567d91c"
server_key_version: "encryptionkey@0"
I20260806 03:55:59.295197 13509 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b"
format_stamp: "Formatted at 2026-08-06 03:55:59 on dist-test-slave-bqcl"
server_key: "f52347500d1e8a4a20b416a5e12daa5c"
server_key_iv: "f6fd730d53223bebef1af07d5567d91c"
server_key_version: "encryptionkey@0"
I20260806 03:55:59.296411 13509 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:55:59.297037 13528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:55:59.297194 13509 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:55:59.297278 13509 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b"
format_stamp: "Formatted at 2026-08-06 03:55:59 on dist-test-slave-bqcl"
server_key: "f52347500d1e8a4a20b416a5e12daa5c"
server_key_iv: "f6fd730d53223bebef1af07d5567d91c"
server_key_version: "encryptionkey@0"
I20260806 03:55:59.297338 13509 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:55:59.300531 13509 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:55:59.302284 13509 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:55:59.302423 13509 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:55:59.306264 13509 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:45341
I20260806 03:55:59.306308 13598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:45341 every 8 connection(s)
I20260806 03:55:59.306607 13509 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:55:59.307560 13600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:55:59.309695 13600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b: Bootstrap starting.
I20260806 03:55:59.310268 13600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b: Neither blocks nor log segments found. Creating new log.
I20260806 03:55:59.310515 13600 log.cc:824] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b: Log is configured to *not* fsync() on all Append() calls
I20260806 03:55:59.311124 13600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b: No bootstrap required, opened a new log
I20260806 03:55:59.312634 13600 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45341 } }
I20260806 03:55:59.312770 13600 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:55:59.312803 13600 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9ebe4b6c5424f3f9c2878d16d16b65b, State: Initialized, Role: FOLLOWER
I20260806 03:55:59.312933 13600 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45341 } }
I20260806 03:55:59.312991 13600 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:55:59.313011 13600 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:55:59.313056 13600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:55:59.313820 13600 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45341 } }
I20260806 03:55:59.313912 13600 leader_election.cc:302] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9ebe4b6c5424f3f9c2878d16d16b65b; no voters:
I20260806 03:55:59.314056 13600 leader_election.cc:288] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:55:59.314090 13604 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:55:59.314289 13604 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [term 1 LEADER]: Becoming Leader. State: Replica: c9ebe4b6c5424f3f9c2878d16d16b65b, State: Running, Role: LEADER
I20260806 03:55:59.314313 13600 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:55:59.314446 13604 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45341 } }
I20260806 03:55:59.314527 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13509
I20260806 03:55:59.314633 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:55:59.314860 12349 external_mini_cluster.cc:1428] Setting key df096d7a2734a0600a9e3c8fcb078076
I20260806 03:55:59.315131 13606 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9ebe4b6c5424f3f9c2878d16d16b65b. Latest consensus state: current_term: 1 leader_uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45341 } } }
I20260806 03:55:59.315238 13606 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:59.315555 13605 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9ebe4b6c5424f3f9c2878d16d16b65b" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 45341 } } }
I20260806 03:55:59.315616 13620 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:55:59.315659 13605 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b [sys.catalog]: This master's current role is: LEADER
I20260806 03:55:59.316083 13620 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:55:59.317561 13620 catalog_manager.cc:1347] Generated new cluster ID: d38e803e685a47249241d69139780d29
I20260806 03:55:59.317610 13620 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:55:59.317833 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:59.315504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42743 (local address 127.12.15.126:45341)
0806 03:55:59.316258 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:59.316268 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:59.316298 (+ 30us) server_negotiation.cc:407] Connection header received
0806 03:55:59.316515 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:59.316522 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:59.316553 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:59.316694 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:59.317394 (+ 700us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:59.317398 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:59.317398 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:59.317402 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:59.317452 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:59.317467 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:59.317544 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:59.317623 (+ 79us) server_negotiation.cc:299] Negotiation successful
0806 03:55:59.317665 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":696,"thread_start_us":64,"threads_started":1}
I20260806 03:55:59.318714 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:55:59.321208 13633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:55:59.320196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58911 (local address 127.12.15.126:45341)
0806 03:55:59.320229 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:55:59.320231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:55:59.320355 (+ 124us) server_negotiation.cc:407] Connection header received
0806 03:55:59.320420 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:55:59.320421 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:55:59.320426 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:55:59.320468 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:55:59.321016 (+ 548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:55:59.321017 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:55:59.321017 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:55:59.321020 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:55:59.321033 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:55:59.321037 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:55:59.321077 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0806 03:55:59.321130 (+ 53us) server_negotiation.cc:299] Negotiation successful
0806 03:55:59.321141 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:55:59.329957 13620 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:55:59.330192 13620 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:55:59.346995 13620 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9ebe4b6c5424f3f9c2878d16d16b65b: Generated new TSK 0
I20260806 03:55:59.347226 13620 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:55:59.348912 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13509
2026-08-06T03:55:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:55:59 dist-test-slave-bqcl krb5kdc[13647](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:55:59 dist-test-slave-bqcl krb5kdc[13647](info): set up 2 sockets
Aug 06 03:55:59 dist-test-slave-bqcl krb5kdc[13647](info): commencing operation
krb5kdc: starting...
W20260806 03:56:00.496143 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:56:00 dist-test-slave-bqcl krb5kdc[13647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:56:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:56:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:56:00.613708 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35671
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:46025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:35671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:56:00.694779 13673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:00.694983 13673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:00.695014 13673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:00.696120 13673 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:00.696161 13673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:00.696180 13673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:00.696205 13673 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:00.696228 13673 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:00.697933 13673 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:46025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:35671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:00.698217 13673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:00.698431 13673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:56:00.703960 13679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:00.703971 13686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:00.704223 13680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:00.704378 13673 server_base.cc:1034] running on GCE node
I20260806 03:56:00.704653 13673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:00.704969 13673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:00.706183 13673 hybrid_clock.cc:648] HybridClock initialized: now 1785988560706164 us; error 34 us; skew 500 ppm
Aug 06 03:56:00 dist-test-slave-bqcl krb5kdc[13647](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988560, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:00.708576 13673 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:56:00.708972 13673 webserver.cc:460] Webserver started at http://127.12.15.126:34623/ using document root <none> and password file <none>
I20260806 03:56:00.709184 13673 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:00.709232 13673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:00.709353 13673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:00.710386 13673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "2f4cfb9e3f4442c8b67cdc18438b17df"
format_stamp: "Formatted at 2026-08-06 03:56:00 on dist-test-slave-bqcl"
server_key: "9f123923b25a793cdcc500b5ddac1c56"
server_key_iv: "349b0853d519ad780d49a72f12e4d535"
server_key_version: "encryptionkey@0"
I20260806 03:56:00.710718 13673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "2f4cfb9e3f4442c8b67cdc18438b17df"
format_stamp: "Formatted at 2026-08-06 03:56:00 on dist-test-slave-bqcl"
server_key: "9f123923b25a793cdcc500b5ddac1c56"
server_key_iv: "349b0853d519ad780d49a72f12e4d535"
server_key_version: "encryptionkey@0"
I20260806 03:56:00.711941 13673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260806 03:56:00.712611 13694 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:00.712766 13673 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:56:00.712852 13673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "2f4cfb9e3f4442c8b67cdc18438b17df"
format_stamp: "Formatted at 2026-08-06 03:56:00 on dist-test-slave-bqcl"
server_key: "9f123923b25a793cdcc500b5ddac1c56"
server_key_iv: "349b0853d519ad780d49a72f12e4d535"
server_key_version: "encryptionkey@0"
I20260806 03:56:00.712909 13673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:00.725529 13673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:00.727293 13673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:00.727432 13673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:00.731254 13673 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:35671
I20260806 03:56:00.731288 13765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:35671 every 8 connection(s)
I20260806 03:56:00.731614 13673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:56:00.732458 13766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:00.734704 13766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df: Bootstrap starting.
I20260806 03:56:00.735266 13766 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:00.735517 13766 log.cc:824] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:00.736150 13766 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df: No bootstrap required, opened a new log
I20260806 03:56:00.737677 13766 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cfb9e3f4442c8b67cdc18438b17df" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35671 } }
I20260806 03:56:00.737816 13766 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:56:00.737850 13766 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f4cfb9e3f4442c8b67cdc18438b17df, State: Initialized, Role: FOLLOWER
I20260806 03:56:00.737984 13766 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cfb9e3f4442c8b67cdc18438b17df" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35671 } }
I20260806 03:56:00.738044 13766 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:56:00.738065 13766 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:56:00.738107 13766 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:56:00.738343 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13673
I20260806 03:56:00.738456 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:56:00.738648 12349 external_mini_cluster.cc:1428] Setting key b538130998705316f6ef2a9ff786367c
I20260806 03:56:00.739138 13766 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cfb9e3f4442c8b67cdc18438b17df" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35671 } }
I20260806 03:56:00.739259 13766 leader_election.cc:302] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2f4cfb9e3f4442c8b67cdc18438b17df; no voters:
I20260806 03:56:00.739420 13766 leader_election.cc:288] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:56:00.739449 13773 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:56:00.739609 13773 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [term 1 LEADER]: Becoming Leader. State: Replica: 2f4cfb9e3f4442c8b67cdc18438b17df, State: Running, Role: LEADER
I20260806 03:56:00.739662 13766 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:56:00.739704 13773 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cfb9e3f4442c8b67cdc18438b17df" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35671 } }
I20260806 03:56:00.740056 13773 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2f4cfb9e3f4442c8b67cdc18438b17df. Latest consensus state: current_term: 1 leader_uuid: "2f4cfb9e3f4442c8b67cdc18438b17df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cfb9e3f4442c8b67cdc18438b17df" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35671 } } }
I20260806 03:56:00.740119 13773 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:00.740334 13787 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:56:00.740797 13775 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2f4cfb9e3f4442c8b67cdc18438b17df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2f4cfb9e3f4442c8b67cdc18438b17df" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35671 } } }
I20260806 03:56:00.740934 13775 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:00.741003 13787 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:56:00.742573 13787 catalog_manager.cc:1347] Generated new cluster ID: cc034718a983405eb64ea67fa2e63d07
I20260806 03:56:00.742628 13787 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 06 03:56:00 dist-test-slave-bqcl krb5kdc[13647](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988560, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:56:00.748652 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:00.739134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59213 (local address 127.12.15.126:35671)
0806 03:56:00.739319 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:00.739329 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:00.739366 (+ 37us) server_negotiation.cc:407] Connection header received
0806 03:56:00.739487 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:00.739495 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:00.739553 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:00.739713 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:00.740497 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:00.741964 (+ 1467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:00.742932 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:00.743352 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:00.743460 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:00.745696 (+ 2236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:00.745700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:00.745700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:00.745713 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:00.747262 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:00.747584 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:00.747587 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:00.747587 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:00.747688 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:00.747867 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:00.747868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:00.747869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:00.748099 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:00.748267 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:00.748335 (+ 68us) server_negotiation.cc:299] Negotiation successful
0806 03:56:00.748468 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":86,"threads_started":1}
I20260806 03:56:00.749552 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:56:00.759473 13772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:00.750989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54465 (local address 127.12.15.126:35671)
0806 03:56:00.751022 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:00.751024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:00.754286 (+ 3262us) server_negotiation.cc:407] Connection header received
0806 03:56:00.754343 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:00.754344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:00.754356 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:00.754399 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:00.754848 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:00.755315 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:00.755988 (+ 673us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:00.756196 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:00.756251 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:00.757265 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:00.757266 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:00.757267 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:00.757273 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:00.758508 (+ 1235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:00.758810 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:00.758811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:00.758812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:00.758866 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:00.759022 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:00.759023 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:00.759023 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:00.759126 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:00.759221 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:00.759318 (+ 97us) server_negotiation.cc:299] Negotiation successful
0806 03:56:00.759389 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:56:00.775802 13787 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:56:00.776330 13787 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:56:00.778550 13787 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2f4cfb9e3f4442c8b67cdc18438b17df: Generated new TSK 0
I20260806 03:56:00.778715 13787 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:56:00.781957 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13673
2026-08-06T03:56:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1429 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:56:00 dist-test-slave-bqcl krb5kdc[13817](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:56:00 dist-test-slave-bqcl krb5kdc[13817](info): set up 2 sockets
Aug 06 03:56:00 dist-test-slave-bqcl krb5kdc[13817](info): commencing operation
krb5kdc: starting...
W20260806 03:56:01.918675 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.113s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:56:01 dist-test-slave-bqcl krb5kdc[13817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988561, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:56:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:56:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:56:02.038964 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35497
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:38183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:35497
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:56:02.120380 13841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:02.120595 13841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:02.120627 13841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:02.121778 13841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:02.121822 13841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:02.121842 13841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:02.121857 13841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:02.121872 13841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:02.123353 13841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:38183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:35497
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:35497
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:02.123603 13841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:02.123811 13841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:56:02.128523 13848 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:02.128513 13853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:02.128496 13847 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:02.128544 13841 server_base.cc:1034] running on GCE node
I20260806 03:56:02.129017 13841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:02.129308 13841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:02.130474 13841 hybrid_clock.cc:648] HybridClock initialized: now 1785988562130451 us; error 32 us; skew 500 ppm
Aug 06 03:56:02 dist-test-slave-bqcl krb5kdc[13817](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988562, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:02.135890 13841 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:56:02.136336 13841 webserver.cc:460] Webserver started at http://127.12.15.126:44881/ using document root <none> and password file <none>
I20260806 03:56:02.136561 13841 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:02.136613 13841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:02.136741 13841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:02.137804 13841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "29597090b8cf427194486a364101e3f3"
format_stamp: "Formatted at 2026-08-06 03:56:02 on dist-test-slave-bqcl"
server_key: "3cf581699dc1bcc08e37ac14a821cbef"
server_key_iv: "28ecb84e068e43a180fe0e568baa1122"
server_key_version: "encryptionkey@0"
I20260806 03:56:02.138188 13841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "29597090b8cf427194486a364101e3f3"
format_stamp: "Formatted at 2026-08-06 03:56:02 on dist-test-slave-bqcl"
server_key: "3cf581699dc1bcc08e37ac14a821cbef"
server_key_iv: "28ecb84e068e43a180fe0e568baa1122"
server_key_version: "encryptionkey@0"
I20260806 03:56:02.139447 13841 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:56:02.140281 13865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:02.140439 13841 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:56:02.140532 13841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "29597090b8cf427194486a364101e3f3"
format_stamp: "Formatted at 2026-08-06 03:56:02 on dist-test-slave-bqcl"
server_key: "3cf581699dc1bcc08e37ac14a821cbef"
server_key_iv: "28ecb84e068e43a180fe0e568baa1122"
server_key_version: "encryptionkey@0"
I20260806 03:56:02.140625 13841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:02.167523 13841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:02.169886 13841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:02.170042 13841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:02.173990 13841 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:35497
I20260806 03:56:02.174059 13955 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:35497 every 8 connection(s)
I20260806 03:56:02.174399 13841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:56:02.175251 13956 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:02.177517 13956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3: Bootstrap starting.
I20260806 03:56:02.178053 13956 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:02.178359 13956 log.cc:824] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:02.179035 13956 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3: No bootstrap required, opened a new log
I20260806 03:56:02.180583 13956 raft_consensus.cc:348] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29597090b8cf427194486a364101e3f3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35497 } }
I20260806 03:56:02.180730 13956 raft_consensus.cc:374] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:56:02.180752 13956 raft_consensus.cc:729] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 29597090b8cf427194486a364101e3f3, State: Initialized, Role: FOLLOWER
I20260806 03:56:02.180866 13956 consensus_queue.cc:260] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29597090b8cf427194486a364101e3f3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35497 } }
I20260806 03:56:02.180917 13956 raft_consensus.cc:388] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:56:02.180938 13956 raft_consensus.cc:482] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:56:02.180967 13956 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:56:02.181766 13956 raft_consensus.cc:504] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29597090b8cf427194486a364101e3f3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35497 } }
I20260806 03:56:02.181861 13956 leader_election.cc:302] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 29597090b8cf427194486a364101e3f3; no voters:
I20260806 03:56:02.181995 13956 leader_election.cc:288] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:56:02.182044 13964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:56:02.182238 13964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [term 1 LEADER]: Becoming Leader. State: Replica: 29597090b8cf427194486a364101e3f3, State: Running, Role: LEADER
I20260806 03:56:02.182284 13956 sys_catalog.cc:564] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:56:02.182345 13964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29597090b8cf427194486a364101e3f3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35497 } }
I20260806 03:56:02.182734 13965 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "29597090b8cf427194486a364101e3f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29597090b8cf427194486a364101e3f3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35497 } } }
I20260806 03:56:02.182744 13969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 29597090b8cf427194486a364101e3f3. Latest consensus state: current_term: 1 leader_uuid: "29597090b8cf427194486a364101e3f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "29597090b8cf427194486a364101e3f3" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 35497 } } }
I20260806 03:56:02.182883 13965 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:02.182893 13969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3 [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:02.183104 13980 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:56:02.183820 13980 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:56:02.184195 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 13841
I20260806 03:56:02.184305 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:56:02.184482 12349 external_mini_cluster.cc:1428] Setting key 16dfab43b7eb96eaa41d863e820be1c5
I20260806 03:56:02.185523 13980 catalog_manager.cc:1347] Generated new cluster ID: c9d839c907964deaba5b84c4c71a0b7d
I20260806 03:56:02.185575 13980 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:56:02.189689 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:02.184973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36121 (local address 127.12.15.126:35497)
0806 03:56:02.185577 (+ 604us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:02.185584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:02.185614 (+ 30us) server_negotiation.cc:407] Connection header received
0806 03:56:02.185727 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:02.185732 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:02.185775 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:02.185898 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:02.186161 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:02.187112 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:02.187887 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:02.188308 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:02.188407 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:02.189060 (+ 653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:02.189064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:02.189064 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:56:02.189235 (+ 171us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:02.189267 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:56:02.189279 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:02.189336 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:02.189410 (+ 74us) server_negotiation.cc:299] Negotiation successful
0806 03:56:02.189476 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":81,"threads_started":1}
I20260806 03:56:02.190802 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:56:02.194761 14001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:02.192258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34167 (local address 127.12.15.126:35497)
0806 03:56:02.192295 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:02.192296 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:02.192303 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:56:02.192320 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:02.192321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:02.192330 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:02.192376 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:02.192506 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:02.192976 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:02.193643 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:02.193856 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:02.193912 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:02.194535 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:02.194536 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:02.194537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:56:02.194543 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:02.194558 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:56:02.194562 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:02.194606 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:02.194653 (+ 47us) server_negotiation.cc:299] Negotiation successful
0806 03:56:02.194684 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:56:02.224887 13980 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:56:02.225466 13980 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:56:02.227295 13980 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 29597090b8cf427194486a364101e3f3: Generated new TSK 0
I20260806 03:56:02.227504 13980 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:56:02.232389 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 13841
2026-08-06T03:56:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1449 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:56:02 dist-test-slave-bqcl krb5kdc[14015](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:56:02 dist-test-slave-bqcl krb5kdc[14015](info): set up 2 sockets
Aug 06 03:56:02 dist-test-slave-bqcl krb5kdc[14015](info): commencing operation
krb5kdc: starting...
W20260806 03:56:03.361389 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.106s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:56:03 dist-test-slave-bqcl krb5kdc[14015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988563, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:56:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:56:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:56:03.480628 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33317
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:46657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:33317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:56:03.563644 14032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:03.563869 14032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:03.563905 14032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:03.565055 14032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:03.565106 14032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:03.565130 14032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:03.565150 14032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:03.565169 14032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:03.566735 14032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:46657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:33317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:33317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:03.566990 14032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:03.567203 14032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:56:03.571911 14039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:03.571915 14042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:03.571954 14040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:03.572204 14032 server_base.cc:1034] running on GCE node
I20260806 03:56:03.572504 14032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:03.572793 14032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:03.573961 14032 hybrid_clock.cc:648] HybridClock initialized: now 1785988563573944 us; error 26 us; skew 500 ppm
Aug 06 03:56:03 dist-test-slave-bqcl krb5kdc[14015](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988563, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:03.576566 14032 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:56:03.577049 14032 webserver.cc:460] Webserver started at http://127.12.15.126:40657/ using document root <none> and password file <none>
I20260806 03:56:03.577258 14032 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:03.577311 14032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:03.577448 14032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:03.578608 14032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "ea6f38fa21664854aecaafc77f7cce3a"
format_stamp: "Formatted at 2026-08-06 03:56:03 on dist-test-slave-bqcl"
server_key: "12bd8de5158fc8ca44ee8901b0b61ce4"
server_key_iv: "48f007288a143035fb00b28d26426d96"
server_key_version: "encryptionkey@0"
I20260806 03:56:03.578965 14032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "ea6f38fa21664854aecaafc77f7cce3a"
format_stamp: "Formatted at 2026-08-06 03:56:03 on dist-test-slave-bqcl"
server_key: "12bd8de5158fc8ca44ee8901b0b61ce4"
server_key_iv: "48f007288a143035fb00b28d26426d96"
server_key_version: "encryptionkey@0"
I20260806 03:56:03.580269 14032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260806 03:56:03.580967 14054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:03.581125 14032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:56:03.581218 14032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "ea6f38fa21664854aecaafc77f7cce3a"
format_stamp: "Formatted at 2026-08-06 03:56:03 on dist-test-slave-bqcl"
server_key: "12bd8de5158fc8ca44ee8901b0b61ce4"
server_key_iv: "48f007288a143035fb00b28d26426d96"
server_key_version: "encryptionkey@0"
I20260806 03:56:03.581284 14032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:03.584810 14032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:03.586748 14032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:03.586886 14032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:03.591087 14032 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:33317
I20260806 03:56:03.591118 14127 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:33317 every 8 connection(s)
I20260806 03:56:03.591531 14032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:56:03.592497 14128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:03.594913 14128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a: Bootstrap starting.
I20260806 03:56:03.595204 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14032
I20260806 03:56:03.595314 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:56:03.595498 12349 external_mini_cluster.cc:1428] Setting key 3897a7cf3fa5e2e06ec4a32b9a9c36ce
I20260806 03:56:03.595525 14128 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:03.595793 14128 log.cc:824] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:03.596745 14128 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a: No bootstrap required, opened a new log
I20260806 03:56:03.598054 14131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:03.595976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39945 (local address 127.12.15.126:33317)
0806 03:56:03.596163 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:03.596173 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:03.596204 (+ 31us) server_negotiation.cc:407] Connection header received
0806 03:56:03.596754 (+ 550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:03.596763 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:03.596800 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:03.596955 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:03.597624 (+ 669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:03.597628 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:03.597628 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:56:03.597632 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:03.597676 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:56:03.597690 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:03.597783 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:03.597813 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 03:56:03.597855 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":63,"threads_started":1}
I20260806 03:56:03.598965 14128 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6f38fa21664854aecaafc77f7cce3a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33317 } }
I20260806 03:56:03.599123 14128 raft_consensus.cc:374] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260806 03:56:03.599119 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:56:03.599154 14128 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea6f38fa21664854aecaafc77f7cce3a, State: Initialized, Role: FOLLOWER
I20260806 03:56:03.599277 14128 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6f38fa21664854aecaafc77f7cce3a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33317 } }
I20260806 03:56:03.599347 14128 raft_consensus.cc:388] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260806 03:56:03.599370 14128 raft_consensus.cc:482] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260806 03:56:03.599416 14128 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:56:03.599944 14128 raft_consensus.cc:504] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6f38fa21664854aecaafc77f7cce3a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33317 } }
I20260806 03:56:03.600034 14128 leader_election.cc:302] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ea6f38fa21664854aecaafc77f7cce3a; no voters:
I20260806 03:56:03.600203 14128 leader_election.cc:288] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260806 03:56:03.600376 14139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:56:03.600462 14128 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:56:03.600529 14139 raft_consensus.cc:686] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [term 1 LEADER]: Becoming Leader. State: Replica: ea6f38fa21664854aecaafc77f7cce3a, State: Running, Role: LEADER
I20260806 03:56:03.600703 14139 consensus_queue.cc:237] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6f38fa21664854aecaafc77f7cce3a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33317 } }
I20260806 03:56:03.601049 14139 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [sys.catalog]: SysCatalogTable state changed. Reason: New leader ea6f38fa21664854aecaafc77f7cce3a. Latest consensus state: current_term: 1 leader_uuid: "ea6f38fa21664854aecaafc77f7cce3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6f38fa21664854aecaafc77f7cce3a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33317 } } }
I20260806 03:56:03.601058 14140 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ea6f38fa21664854aecaafc77f7cce3a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ea6f38fa21664854aecaafc77f7cce3a" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 33317 } } }
I20260806 03:56:03.601145 14139 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:03.601150 14140 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:03.601496 14155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:56:03.602486 14131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:03.600691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44457 (local address 127.12.15.126:33317)
0806 03:56:03.600722 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:03.600724 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:03.601442 (+ 718us) server_negotiation.cc:407] Connection header received
0806 03:56:03.601536 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:03.601537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:03.601541 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:03.601568 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:03.602282 (+ 714us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:03.602283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:03.602284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0806 03:56:03.602288 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:03.602302 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0806 03:56:03.602306 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:03.602353 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:03.602414 (+ 61us) server_negotiation.cc:299] Negotiation successful
0806 03:56:03.602426 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:56:03.602737 14155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:56:03.604364 14155 catalog_manager.cc:1347] Generated new cluster ID: a7201f380c3846c0afaf80c09508d7d1
I20260806 03:56:03.604423 14155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:56:03.616233 14155 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:56:03.616403 14155 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:56:03.621572 14155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ea6f38fa21664854aecaafc77f7cce3a: Generated new TSK 0
I20260806 03:56:03.621755 14155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260806 03:56:03.630918 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14032
2026-08-06T03:56:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1407 ms)
[----------] 18 tests from AuthTokenIssuingTest (25372 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:56:03 dist-test-slave-bqcl krb5kdc[14174](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:56:03 dist-test-slave-bqcl krb5kdc[14174](info): set up 2 sockets
Aug 06 03:56:03 dist-test-slave-bqcl krb5kdc[14174](info): commencing operation
krb5kdc: starting...
W20260806 03:56:04.788631 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:56:04 dist-test-slave-bqcl krb5kdc[14174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:56:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:56:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:56:04.900888 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36399
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:42359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:36399,127.12.15.125:46045,127.12.15.124:35871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:56:04.981684 14197 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:56:04.981901 14197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:04.981917 14197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:04.981937 14197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:04.983119 14197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:04.983163 14197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:04.983175 14197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:04.983199 14197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:04.983222 14197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:04.984771 14197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:42359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:36399,127.12.15.125:46045,127.12.15.124:35871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:36399
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:04.985023 14197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:04.985250 14197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:56:04.990017 14203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:04.990001 14205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:04.990041 14197 server_base.cc:1034] running on GCE node
W20260806 03:56:04.989987 14207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:04.990484 14197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:04.990784 14197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:04.991921 14197 hybrid_clock.cc:648] HybridClock initialized: now 1785988564991907 us; error 29 us; skew 500 ppm
Aug 06 03:56:04 dist-test-slave-bqcl krb5kdc[14174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988564, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:04.994271 14197 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:56:04.994724 14197 webserver.cc:460] Webserver started at http://127.12.15.126:36889/ using document root <none> and password file <none>
I20260806 03:56:04.994946 14197 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:04.994994 14197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:04.995123 14197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:04.996158 14197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "4ef19a51d989452ba9ba6228fc30bd19"
format_stamp: "Formatted at 2026-08-06 03:56:04 on dist-test-slave-bqcl"
server_key: "0b4eac7b6447c12c3f1636925a8b98a1"
server_key_iv: "65531635760f0d5465d71b19dfd4a487"
server_key_version: "encryptionkey@0"
I20260806 03:56:04.996507 14197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "4ef19a51d989452ba9ba6228fc30bd19"
format_stamp: "Formatted at 2026-08-06 03:56:04 on dist-test-slave-bqcl"
server_key: "0b4eac7b6447c12c3f1636925a8b98a1"
server_key_iv: "65531635760f0d5465d71b19dfd4a487"
server_key_version: "encryptionkey@0"
I20260806 03:56:04.997741 14197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:56:04.998445 14220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:04.998592 14197 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:04.998687 14197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "4ef19a51d989452ba9ba6228fc30bd19"
format_stamp: "Formatted at 2026-08-06 03:56:04 on dist-test-slave-bqcl"
server_key: "0b4eac7b6447c12c3f1636925a8b98a1"
server_key_iv: "65531635760f0d5465d71b19dfd4a487"
server_key_version: "encryptionkey@0"
I20260806 03:56:04.998764 14197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:05.020147 14197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:05.021890 14197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:05.022027 14197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:05.025930 14197 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:36399
I20260806 03:56:05.025983 14291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:36399 every 8 connection(s)
I20260806 03:56:05.026330 14197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:56:05.027321 14292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:05.028855 14292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988564, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:56:05.035409 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14197
I20260806 03:56:05.035514 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:56:05.035668 12349 external_mini_cluster.cc:1428] Setting key 216486514e6deb06153c1cb870a1b28b
I20260806 03:56:05.037844 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.029508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36399 (local address 127.0.0.1:39388)
0806 03:56:05.029695 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.029728 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.030220 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.030446 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.030448 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.030685 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.030843 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.030847 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.031789 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.031790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.032452 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.032453 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.032916 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.032917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.032984 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.033408 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.033417 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.034775 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.036355 (+ 1580us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.036364 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.036366 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.036611 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.036815 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.036815 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.036816 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.036924 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.037218 (+ 294us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.037224 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.037302 (+ 78us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.037402 (+ 100us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.037638 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":96,"spinlock_wait_cycles":8448,"thread_start_us":111,"threads_started":1}
I20260806 03:56:05.037846 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.029525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39388 (local address 127.12.15.126:36399)
0806 03:56:05.029678 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.029688 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.029794 (+ 106us) server_negotiation.cc:407] Connection header received
0806 03:56:05.030341 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.030342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.030359 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.030433 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.030909 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.031723 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.032508 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.032802 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.032901 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.034861 (+ 1960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.034868 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.034869 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.034875 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.036276 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.036680 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.036681 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.036682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.036763 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.036979 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.036979 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.036979 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.037163 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.037221 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.037445 (+ 224us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.037570 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":72,"threads_started":1}
I20260806 03:56:05.038561 14292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:05.038862 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.038759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58184)
0806 03:56:05.038801 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.038843 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 03:56:05.038983 14226 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
I20260806 03:56:05.039367 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.039287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58192)
0806 03:56:05.039322 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.039346 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 03:56:05.039460 14292 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:46045: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.12.15.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab.
Entry for principal kudu/127.12.15.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab.
I20260806 03:56:05.058079 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.125:46045
--webserver_interface=127.12.15.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab
--principal=kudu/127.12.15.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:42359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:36399,127.12.15.125:46045,127.12.15.124:35871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:56:05.088665 14292 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } attempt: 1
I20260806 03:56:05.089124 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.089008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58202)
0806 03:56:05.089060 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.089092 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 03:56:05.089504 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.089417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58204)
0806 03:56:05.089460 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.089486 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260806 03:56:05.089605 14292 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:46045: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
W20260806 03:56:05.138207 14304 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:56:05.138435 14304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:05.138461 14304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:05.138485 14304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:05.139606 14304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:05.139645 14304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:05.139664 14304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:05.139680 14304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:05.139695 14304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:05.141167 14304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:42359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:36399,127.12.15.125:46045,127.12.15.124:35871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab
--principal=kudu/127.12.15.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.125:46045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.15.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:05.141419 14304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:05.141630 14304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:56:05.146303 14320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:05.146373 14304 server_base.cc:1034] running on GCE node
W20260806 03:56:05.146353 14317 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:05.146318 14316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:05.146781 14304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:05.147068 14304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:05.148216 14304 hybrid_clock.cc:648] HybridClock initialized: now 1785988565148194 us; error 32 us; skew 500 ppm
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988565, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:05.150635 14304 init.cc:377] Logged in from keytab as kudu/127.12.15.125@KRBTEST.COM (short username kudu)
I20260806 03:56:05.151052 14304 webserver.cc:460] Webserver started at http://127.12.15.125:42687/ using document root <none> and password file <none>
I20260806 03:56:05.151252 14304 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:05.151300 14304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:05.151428 14304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:05.152501 14304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/data/instance:
uuid: "6b605391bde0422d8d71a3093a71896e"
format_stamp: "Formatted at 2026-08-06 03:56:05 on dist-test-slave-bqcl"
server_key: "61fe12e58407610ba0af83702e39e4d3"
server_key_iv: "ba01a057ac076c0c7ea2e2d37e3aa368"
server_key_version: "encryptionkey@0"
I20260806 03:56:05.152840 14304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/wal/instance:
uuid: "6b605391bde0422d8d71a3093a71896e"
format_stamp: "Formatted at 2026-08-06 03:56:05 on dist-test-slave-bqcl"
server_key: "61fe12e58407610ba0af83702e39e4d3"
server_key_iv: "ba01a057ac076c0c7ea2e2d37e3aa368"
server_key_version: "encryptionkey@0"
I20260806 03:56:05.154048 14304 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:56:05.154719 14329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:05.154871 14304 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:05.154955 14304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/wal
uuid: "6b605391bde0422d8d71a3093a71896e"
format_stamp: "Formatted at 2026-08-06 03:56:05 on dist-test-slave-bqcl"
server_key: "61fe12e58407610ba0af83702e39e4d3"
server_key_iv: "ba01a057ac076c0c7ea2e2d37e3aa368"
server_key_version: "encryptionkey@0"
I20260806 03:56:05.155014 14304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:05.157749 14292 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } attempt: 2
I20260806 03:56:05.158141 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.157971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58214)
0806 03:56:05.158037 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.158082 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260806 03:56:05.158491 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.158400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58222)
0806 03:56:05.158440 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.158468 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 03:56:05.158586 14292 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:46045: Network error: Client connection negotiation failed: client connection to 127.12.15.125:46045: connect: Connection refused (error 111)
I20260806 03:56:05.163228 14304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:05.164825 14304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:05.164928 14304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:05.168751 14304 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.125:46045
I20260806 03:56:05.168799 14416 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.125:46045 every 8 connection(s)
I20260806 03:56:05.169093 14304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
I20260806 03:56:05.169981 14417 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:05.171451 14417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:05.172559 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14304
I20260806 03:56:05.172663 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-1/wal/instance
I20260806 03:56:05.172843 12349 external_mini_cluster.cc:1428] Setting key 4bd438cfae2d4b218a85a95a0413cef9
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988565, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:56:05.180327 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.172047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39390 (local address 127.12.15.126:36399)
0806 03:56:05.172095 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.172097 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.172299 (+ 202us) server_negotiation.cc:407] Connection header received
0806 03:56:05.172772 (+ 473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.172773 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.172788 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.172851 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.173435 (+ 584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.174079 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.175314 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.175513 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.175563 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.177782 (+ 2219us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.177784 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.177784 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.177790 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.179064 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.179437 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.179438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.179439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.179495 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.179722 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.179723 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.179723 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.179834 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.179891 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.180147 (+ 256us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.180235 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:56:05.180497 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.172031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36399 (local address 127.0.0.1:39390)
0806 03:56:05.172183 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.172219 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.172673 (+ 454us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.172912 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.172916 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.173165 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.173347 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.173351 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.174174 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.174175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.175232 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.175235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.175594 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.175595 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.175757 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.176212 (+ 455us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.176221 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.177692 (+ 1471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.179137 (+ 1445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.179144 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.179147 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.179369 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.179551 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.179552 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.179552 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.179659 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.179891 (+ 232us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.179898 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.180031 (+ 133us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.180119 (+ 88us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.180284 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":71,"threads_started":1}
I20260806 03:56:05.181025 14417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.12.15.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.124@KRBTEST.COM" created.
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988565, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:56:05.187836 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.181272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58238)
0806 03:56:05.181313 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.181323 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.181353 (+ 30us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.181615 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.181617 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.181753 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.181809 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.181811 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.182672 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.182673 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.183158 (+ 485us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.183160 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.183582 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.183583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.183637 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.183988 (+ 351us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.183992 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.185337 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.186924 (+ 1587us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.186925 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.186926 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.187092 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.187308 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.187308 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.187308 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.187353 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.187623 (+ 270us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.187623 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.187654 (+ 31us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.187697 (+ 43us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.187775 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 03:56:05.187947 14422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.181345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58238 (local address 127.12.15.125:46045)
0806 03:56:05.181485 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.181488 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.181505 (+ 17us) server_negotiation.cc:407] Connection header received
0806 03:56:05.181520 (+ 15us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.181522 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.181531 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.181596 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.181883 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.182597 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.183237 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.183528 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.183573 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.185415 (+ 1842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.185423 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.185424 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.185430 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.186845 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.187174 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.187177 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.187177 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.187249 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.187403 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.187404 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.187404 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.187567 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.187646 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.187739 (+ 93us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.187852 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":116,"thread_start_us":77,"threads_started":1}
I20260806 03:56:05.188503 14417 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:05.188795 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.188689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43350)
0806 03:56:05.188734 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.188777 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260806 03:56:05.188930 14333 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
I20260806 03:56:05.189333 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.189253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43362)
0806 03:56:05.189293 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.189317 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 03:56:05.189435 14417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:35871: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab.
Entry for principal kudu/127.12.15.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab.
I20260806 03:56:05.199204 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.124:35871
--webserver_interface=127.12.15.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab
--principal=kudu/127.12.15.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:42359
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:36399,127.12.15.125:46045,127.12.15.124:35871
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:56:05.238679 14417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } attempt: 1
I20260806 03:56:05.239212 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.239057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43376)
0806 03:56:05.239116 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.239165 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 03:56:05.239521 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.239440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43388)
0806 03:56:05.239475 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.239500 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 03:56:05.239611 14417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:35871: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
I20260806 03:56:05.249714 14292 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } attempt: 3
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988564, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:56:05.256886 14422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.250101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58244 (local address 127.12.15.125:46045)
0806 03:56:05.250171 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.250173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.250227 (+ 54us) server_negotiation.cc:407] Connection header received
0806 03:56:05.250255 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.250256 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.250271 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.250319 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.250650 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.251444 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.252113 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.252460 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.252529 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.254573 (+ 2044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.254575 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.254575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.254582 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.255849 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.256237 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.256237 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.256238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.256284 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.256448 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.256449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.256449 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.256557 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.256604 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.256748 (+ 144us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.256812 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 03:56:05.256937 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.250031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58244)
0806 03:56:05.250087 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.250101 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.250170 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.250338 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.250339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.250506 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.250581 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.250583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.251521 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.251521 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.252044 (+ 523us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.252045 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.252524 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.252525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.252579 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.253028 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.253033 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.254481 (+ 1448us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.255930 (+ 1449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.255931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.255932 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.256160 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.256334 (+ 174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.256334 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.256335 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.256401 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.256608 (+ 207us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.256608 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.256662 (+ 54us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.256729 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.256830 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 03:56:05.257391 14292 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:05.257750 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.257635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43394)
0806 03:56:05.257680 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.257719 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 03:56:05.258092 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.258016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43402)
0806 03:56:05.258052 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.258076 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
W20260806 03:56:05.258196 14292 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:35871: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
W20260806 03:56:05.283321 14427 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:56:05.283531 14427 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:05.283547 14427 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:05.283567 14427 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:05.284682 14427 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:05.284751 14427 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:05.284768 14427 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:05.284783 14427 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:05.284807 14427 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:05.286315 14427 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:42359
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:36399,127.12.15.125:46045,127.12.15.124:35871
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab
--principal=kudu/127.12.15.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.124:35871
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.15.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14427
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:05.286547 14427 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:05.286770 14427 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260806 03:56:05.289351 14292 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } attempt: 1
I20260806 03:56:05.289741 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.289626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43406)
0806 03:56:05.289678 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.289715 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:56:05.290160 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.290038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43420)
0806 03:56:05.290081 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.290108 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 03:56:05.290324 14292 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:35871: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
W20260806 03:56:05.291425 14435 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:05.291455 14433 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:05.291460 14437 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:05.291477 14427 server_base.cc:1034] running on GCE node
I20260806 03:56:05.291816 14427 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:05.292101 14427 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:05.293241 14427 hybrid_clock.cc:648] HybridClock initialized: now 1785988565293227 us; error 26 us; skew 500 ppm
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988565, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:05.295593 14427 init.cc:377] Logged in from keytab as kudu/127.12.15.124@KRBTEST.COM (short username kudu)
I20260806 03:56:05.296020 14427 webserver.cc:460] Webserver started at http://127.12.15.124:41187/ using document root <none> and password file <none>
I20260806 03:56:05.296224 14427 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:05.296271 14427 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:05.296392 14427 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:05.297401 14427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/data/instance:
uuid: "b84789f4e593461191cbca3f5ca17cc8"
format_stamp: "Formatted at 2026-08-06 03:56:05 on dist-test-slave-bqcl"
server_key: "e06b4fdabd9b672438a2a3d356c3e8c1"
server_key_iv: "a4d36732ab2c9481fe9813efcbab36ed"
server_key_version: "encryptionkey@0"
I20260806 03:56:05.297726 14427 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/wal/instance:
uuid: "b84789f4e593461191cbca3f5ca17cc8"
format_stamp: "Formatted at 2026-08-06 03:56:05 on dist-test-slave-bqcl"
server_key: "e06b4fdabd9b672438a2a3d356c3e8c1"
server_key_iv: "a4d36732ab2c9481fe9813efcbab36ed"
server_key_version: "encryptionkey@0"
I20260806 03:56:05.299000 14427 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260806 03:56:05.299625 14447 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:05.299773 14427 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260806 03:56:05.299855 14427 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/wal
uuid: "b84789f4e593461191cbca3f5ca17cc8"
format_stamp: "Formatted at 2026-08-06 03:56:05 on dist-test-slave-bqcl"
server_key: "e06b4fdabd9b672438a2a3d356c3e8c1"
server_key_iv: "a4d36732ab2c9481fe9813efcbab36ed"
server_key_version: "encryptionkey@0"
I20260806 03:56:05.299913 14427 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:05.307758 14417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } attempt: 2
I20260806 03:56:05.308148 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.308011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43426)
0806 03:56:05.308070 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.308113 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260806 03:56:05.308462 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.308389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43436)
0806 03:56:05.308423 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.308445 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 03:56:05.308549 14417 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:35871: Network error: Client connection negotiation failed: client connection to 127.12.15.124:35871: connect: Connection refused (error 111)
I20260806 03:56:05.330963 14427 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:05.332733 14427 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:05.332867 14427 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:05.336877 14427 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.124:35871
I20260806 03:56:05.336920 14537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.124:35871 every 8 connection(s)
I20260806 03:56:05.337275 14427 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
I20260806 03:56:05.338356 14539 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:05.339928 14539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:05.344620 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14427
I20260806 03:56:05.344703 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1785988295328500-12349-0/minicluster-data/master-2/wal/instance
I20260806 03:56:05.344857 12349 external_mini_cluster.cc:1428] Setting key ca4165f097b14d0e128889f97ce9c2eb
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988565, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:56:05.348243 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.340605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39402 (local address 127.12.15.126:36399)
0806 03:56:05.340685 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.340687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.340858 (+ 171us) server_negotiation.cc:407] Connection header received
0806 03:56:05.341315 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.341316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.341325 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.341375 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.341923 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.342424 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.343328 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.343514 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.343566 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.345614 (+ 2048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.345615 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.345616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.345624 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.346990 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.347429 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.347430 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.347430 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.347493 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.347699 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.347700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.347700 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.347828 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.347889 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.348114 (+ 225us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.348178 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260806 03:56:05.348457 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.340576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:36399 (local address 127.0.0.1:39402)
0806 03:56:05.340742 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.340781 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.341238 (+ 457us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.341429 (+ 191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.341432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.341675 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.341865 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.341869 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.342481 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.342482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.343260 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.343262 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.343579 (+ 317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.343579 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.343674 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.344086 (+ 412us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.344093 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.345547 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.347069 (+ 1522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.347076 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.347078 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.347371 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.347553 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.347554 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.347554 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.347643 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.347881 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.347884 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.347992 (+ 108us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.348088 (+ 96us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.348238 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":85,"threads_started":1}
I20260806 03:56:05.348985 14539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988565, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:56:05.353549 14548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.345211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43440 (local address 127.12.15.124:35871)
0806 03:56:05.345369 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.345373 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.345457 (+ 84us) server_negotiation.cc:407] Connection header received
0806 03:56:05.345518 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.345519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.345537 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.345599 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.346416 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.347186 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.348054 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.348471 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.348538 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.350906 (+ 2368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.350914 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.350915 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.350921 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.352412 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.352737 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.352740 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.352740 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.352796 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.352971 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.352972 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.352972 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.353140 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.353272 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.353370 (+ 98us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.353445 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":70,"threads_started":1}
I20260806 03:56:05.355394 14422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.349306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58250 (local address 127.12.15.125:46045)
0806 03:56:05.349370 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.349372 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.349380 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:56:05.349402 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.349403 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.349413 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.349455 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.349701 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.350384 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.351028 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.351308 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.351375 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.352967 (+ 1592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.352969 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.352969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.352977 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.354384 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.354723 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.354724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.354724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.354784 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.354944 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.354945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.354945 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.355082 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.355147 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.355274 (+ 127us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.355332 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260806 03:56:05.355425 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.349230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:46045 (local address 127.0.0.1:58250)
0806 03:56:05.349269 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.349278 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.349309 (+ 31us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.349463 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.349464 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.349590 (+ 126us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.349635 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.349636 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.350458 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.350459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.350952 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.350954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.351376 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.351377 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.351435 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.351738 (+ 303us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.351742 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.352886 (+ 1144us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.354441 (+ 1555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.354442 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.354442 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.354642 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.354840 (+ 198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.354840 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.354841 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.354892 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.355135 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.355136 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.355170 (+ 34us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.355245 (+ 75us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.355332 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260806 03:56:05.355738 14539 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988565, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
I20260806 03:56:05.361469 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.354537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39404 (local address 127.12.15.126:36399)
0806 03:56:05.354581 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.354583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.354591 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:56:05.354611 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.354612 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.354622 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.354684 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.355191 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.355823 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.356695 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.356911 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.356962 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.359046 (+ 2084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.359047 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.359048 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.359054 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.360478 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.360788 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.360789 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.360789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.360845 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.361013 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.361014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.361014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.361116 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.361279 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.361309 (+ 30us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.361382 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:56:05.361795 14545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.355956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43452)
0806 03:56:05.355996 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.356006 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.356039 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.356153 (+ 114us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.356153 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.356254 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.356281 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.356282 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.356988 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.356989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.357564 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.357566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.357959 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.357960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.358010 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.358357 (+ 347us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.358362 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.359498 (+ 1136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.360855 (+ 1357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.360856 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.360856 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.361049 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.361225 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.361225 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.361226 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.361289 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.361527 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.361528 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.361587 (+ 59us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.361655 (+ 68us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.361739 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 03:56:05.362221 14548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.355983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43452 (local address 127.12.15.124:35871)
0806 03:56:05.356016 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.356018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.356028 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:56:05.356089 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.356089 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.356098 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.356149 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.356334 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.356925 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.357630 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.357893 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.357942 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.359572 (+ 1630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.359574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.359574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.359581 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.360785 (+ 1204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.361109 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.361110 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.361110 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.361161 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.361345 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.361346 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.361346 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.361464 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.361531 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.362048 (+ 517us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.362108 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260806 03:56:05.363691 14539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8: Bootstrap starting.
I20260806 03:56:05.364593 14539 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:05.364933 14539 log.cc:824] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:05.365459 14292 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } attempt: 2
I20260806 03:56:05.365713 14539 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8: No bootstrap required, opened a new log
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988564, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:56:05.367688 14539 raft_consensus.cc:348] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } }
I20260806 03:56:05.367857 14539 raft_consensus.cc:729] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b84789f4e593461191cbca3f5ca17cc8, State: Initialized, Role: FOLLOWER
I20260806 03:56:05.367942 14539 consensus_queue.cc:260] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } }
I20260806 03:56:05.368261 14539 sys_catalog.cc:564] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:56:05.368211 14551 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.368335 14551 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:05.369468 14422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.362397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58256 (local address 127.12.15.125:46045)
0806 03:56:05.362430 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.362432 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.362441 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:56:05.362517 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.362518 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.362528 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.362573 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.362978 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.363634 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.364258 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.364554 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.364631 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.366773 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.366774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.366774 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.366781 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.368019 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.368575 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.368576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.368576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.368627 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.368822 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.368823 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.368823 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.368948 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.369056 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.369293 (+ 237us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.369379 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988564, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
W20260806 03:56:05.370671 14565 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260806 03:56:05.371009 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260806 03:56:05.371111 14565 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:56:05.372033 14257 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "4ef19a51d989452ba9ba6228fc30bd19"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39404
I20260806 03:56:05.373600 14296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.365705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43464)
0806 03:56:05.365747 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.365755 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.365790 (+ 35us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.365910 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.365911 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.366052 (+ 141us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.366089 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.366091 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.366935 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.366937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.367721 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.367725 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.368206 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.368208 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.368271 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.368583 (+ 312us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.368587 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.369905 (+ 1318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.372564 (+ 2659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.372565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.372566 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.372768 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.372974 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.372975 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.372975 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.373032 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.373329 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.373331 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.373376 (+ 45us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.373420 (+ 44us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.373510 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260806 03:56:05.370741 14427 master_runner.cc:186] Error getting master registration for 127.12.15.126:36399: OK, instance_id { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" instance_seqno: 1785988565024861 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:56:05.373972 14548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.365736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43464 (local address 127.12.15.124:35871)
0806 03:56:05.365769 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.365771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.365781 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:56:05.365839 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.365840 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.365848 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.365904 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.366188 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.366827 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.367826 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.368112 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.368409 (+ 297us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.370017 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.370018 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.370019 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.370025 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.372462 (+ 2437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.372851 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.372852 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.372853 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.372907 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.373095 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.373096 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.373096 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.373247 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.373781 (+ 534us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.373822 (+ 41us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.373898 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:56:05.374024 14427 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:56:05.375394 14292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19: Bootstrap starting.
I20260806 03:56:05.376096 14292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:05.376408 14292 log.cc:824] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:05.377246 14292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19: No bootstrap required, opened a new log
I20260806 03:56:05.378572 14292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } }
I20260806 03:56:05.378711 14292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ef19a51d989452ba9ba6228fc30bd19, State: Initialized, Role: FOLLOWER
I20260806 03:56:05.378798 14292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } }
I20260806 03:56:05.379101 14292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:56:05.379151 14257 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 03:56:05.379066 14570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.379225 14257 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:56:05.379307 14570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:05.380007 14257 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } }
I20260806 03:56:05.380254 14257 leader_election.cc:288] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [CANDIDATE]: Term 1 election: Requested vote from peers 6b605391bde0422d8d71a3093a71896e (127.12.15.125:46045), b84789f4e593461191cbca3f5ca17cc8 (127.12.15.124:35871)
I20260806 03:56:05.380657 14493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ef19a51d989452ba9ba6228fc30bd19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b84789f4e593461191cbca3f5ca17cc8"
I20260806 03:56:05.380771 14493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [term 0 FOLLOWER]: Advancing to term 1
W20260806 03:56:05.381188 14585 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:56:05.381233 14585 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:56:05.381593 14493 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef19a51d989452ba9ba6228fc30bd19 in term 1.
I20260806 03:56:05.381826 14222 leader_election.cc:302] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ef19a51d989452ba9ba6228fc30bd19, b84789f4e593461191cbca3f5ca17cc8; no voters:
I20260806 03:56:05.382000 14570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:56:05.382210 14570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [term 1 LEADER]: Becoming Leader. State: Replica: 4ef19a51d989452ba9ba6228fc30bd19, State: Running, Role: LEADER
I20260806 03:56:05.382295 14570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } }
I20260806 03:56:05.382688 14570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ef19a51d989452ba9ba6228fc30bd19. Latest consensus state: current_term: 1 leader_uuid: "4ef19a51d989452ba9ba6228fc30bd19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.382752 14570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:05.383641 14197 master_runner.cc:186] Error getting master registration for 127.12.15.125:46045: OK, instance_id { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" instance_seqno: 1785988565167726 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:56:05.385746 14197 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:56:05.385865 14597 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:56:05.386052 14597 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:56:05.387817 14493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [term 1 FOLLOWER]: Refusing update from remote peer 4ef19a51d989452ba9ba6228fc30bd19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:56:05.388139 14588 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:56:05.389374 14551 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ef19a51d989452ba9ba6228fc30bd19. Latest consensus state: current_term: 1 leader_uuid: "4ef19a51d989452ba9ba6228fc30bd19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.389462 14551 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:05.390683 14601 mvcc.cc:205] Tried to move back new op lower bound from 7315409163826216960 to 7315409163806527488. Current Snapshot: MvccSnapshot[applied={T|T < 7315409163826216960}]
I20260806 03:56:05.391023 14570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ef19a51d989452ba9ba6228fc30bd19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.391093 14570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:05.391647 14551 sys_catalog.cc:455] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ef19a51d989452ba9ba6228fc30bd19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.391724 14551 sys_catalog.cc:458] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:05.393555 14294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.383001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39410 (local address 127.12.15.126:36399)
0806 03:56:05.383033 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.383035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.383330 (+ 295us) server_negotiation.cc:407] Connection header received
0806 03:56:05.383430 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.383431 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.383441 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.383468 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.384213 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.384890 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.386776 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.387037 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.387097 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.390593 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.390594 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.390594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.390601 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.392332 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.392756 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.392758 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.392758 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.392821 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.393027 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.393027 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.393028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.393152 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.393250 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.393381 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.393469 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 03:56:05.394201 14602 mvcc.cc:205] Tried to move back new op lower bound from 7315409163826216960 to 7315409163806527488. Current Snapshot: MvccSnapshot[applied={T|T < 7315409163826216960}]
I20260806 03:56:05.394510 14597 catalog_manager.cc:1347] Generated new cluster ID: 952c39e68d9f48d8b79a3d059548baf3
I20260806 03:56:05.394564 14597 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:56:05.399683 14417 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } attempt: 3
I20260806 03:56:05.406205 14422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.383342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58264 (local address 127.12.15.125:46045)
0806 03:56:05.383524 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.383525 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.394403 (+ 10878us) server_negotiation.cc:407] Connection header received
0806 03:56:05.394449 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.394450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.394460 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.394504 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.395782 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.396522 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.401129 (+ 4607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.401475 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.401555 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.403068 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.403069 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.403069 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.403077 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.404887 (+ 1810us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.405250 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.405251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.405251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.405313 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.405671 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.405671 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.405672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.405812 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.405916 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.406012 (+ 96us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.406086 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163}
Aug 06 03:56:05 dist-test-slave-bqcl krb5kdc[14174](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988565, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
I20260806 03:56:05.416431 14420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.399941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:35871 (local address 127.0.0.1:43496)
0806 03:56:05.400016 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:05.400025 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:05.400054 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:05.405285 (+ 5231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:05.405286 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:05.405431 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:05.405484 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.405485 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.407107 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.407108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.407843 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:05.407844 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.408295 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.408296 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:05.408344 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.408682 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:05.408686 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:05.410031 (+ 1345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:05.415425 (+ 5394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.415426 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.415427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.415619 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.415821 (+ 202us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:05.415821 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:05.415821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:05.415880 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:05.416142 (+ 262us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:05.416143 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:05.416184 (+ 41us) client_negotiation.cc:769] Sending connection context
0806 03:56:05.416244 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:56:05.416333 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260806 03:56:05.417351 14610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.400109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43496 (local address 127.12.15.124:35871)
0806 03:56:05.403109 (+ 3000us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.403111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.403117 (+ 6us) server_negotiation.cc:407] Connection header received
0806 03:56:05.403134 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.403135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.403154 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.405670 (+ 2516us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.405701 (+ 31us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.407031 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.407920 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.408227 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.408460 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.410115 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.410116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.410116 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.410149 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.415320 (+ 5171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.415693 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.415694 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.415694 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.415761 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.415942 (+ 181us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.415943 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.415943 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.416074 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.416617 (+ 543us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.416661 (+ 44us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.417236 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2972,"thread_start_us":72,"threads_started":1}
I20260806 03:56:05.418751 14417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e: Bootstrap starting.
I20260806 03:56:05.418816 14548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:05.383027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43480 (local address 127.12.15.124:35871)
0806 03:56:05.383062 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.383064 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.393567 (+ 10503us) server_negotiation.cc:407] Connection header received
0806 03:56:05.393745 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.393746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.393756 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.393784 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.394549 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.395266 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.406973 (+ 11707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:05.407289 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:05.407355 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:05.415630 (+ 8275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:05.415631 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:05.415631 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:05.415638 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:05.417141 (+ 1503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.417709 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.417710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.417710 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.417787 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:05.417997 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:05.417998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:05.417998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:05.418159 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:05.418260 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:05.418658 (+ 398us) server_negotiation.cc:299] Negotiation successful
0806 03:56:05.418728 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:56:05.419466 14417 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:05.419737 14417 log.cc:824] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:05.420740 14417 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e: No bootstrap required, opened a new log
I20260806 03:56:05.422581 14417 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } }
I20260806 03:56:05.422710 14417 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b605391bde0422d8d71a3093a71896e, State: Initialized, Role: FOLLOWER
I20260806 03:56:05.422784 14417 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } }
I20260806 03:56:05.423074 14417 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:56:05.423121 14377 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4ef19a51d989452ba9ba6228fc30bd19" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6b605391bde0422d8d71a3093a71896e"
I20260806 03:56:05.423249 14377 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:56:05.424017 14377 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4ef19a51d989452ba9ba6228fc30bd19 in term 1.
I20260806 03:56:05.424496 14614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.424664 14614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:05.424214 14375 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [term 1 FOLLOWER]: Refusing update from remote peer 4ef19a51d989452ba9ba6228fc30bd19: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:56:05.424970 14607 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:56:05.426399 14614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ef19a51d989452ba9ba6228fc30bd19. Latest consensus state: current_term: 1 leader_uuid: "4ef19a51d989452ba9ba6228fc30bd19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.426527 14614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:05.426599 14614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4ef19a51d989452ba9ba6228fc30bd19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.426687 14614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:05.430820 14633 mvcc.cc:205] Tried to move back new op lower bound from 7315409163826216960 to 7315409163806527488. Current Snapshot: MvccSnapshot[applied={T|T < 7315409163826216960}]
I20260806 03:56:05.431022 14597 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:56:05.431643 14588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4ef19a51d989452ba9ba6228fc30bd19" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4ef19a51d989452ba9ba6228fc30bd19" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 36399 } } peers { permanent_uuid: "6b605391bde0422d8d71a3093a71896e" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 46045 } } peers { permanent_uuid: "b84789f4e593461191cbca3f5ca17cc8" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 35871 } } }
I20260806 03:56:05.431743 14588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19 [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:05.432003 14597 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:56:05.434283 14631 catalog_manager.cc:1259] Loaded cluster ID: 952c39e68d9f48d8b79a3d059548baf3
I20260806 03:56:05.434330 14631 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e: loading cluster ID for follower catalog manager: success
I20260806 03:56:05.435490 14631 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e: acquiring CA information for follower catalog manager: success
W20260806 03:56:05.435606 14631 catalog_manager.cc:873] Not found: no TSK found in the system table: failed to prepare follower catalog manager, will retry
I20260806 03:56:05.451669 14597 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ef19a51d989452ba9ba6228fc30bd19: Generated new TSK 0
I20260806 03:56:05.451874 14597 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:56:05.484154 14294 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.482680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.12.15.126:36399)
0806 03:56:05.482720 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.482722 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.483446 (+ 724us) server_negotiation.cc:407] Connection header received
0806 03:56:05.483556 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.483557 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.483570 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.483599 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.484074 (+ 475us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39416: BlockingRecv error: recv got EOF from 127.0.0.1:39416 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 03:56:05.484930 14610 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.482432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43498 (local address 127.12.15.124:35871)
0806 03:56:05.482480 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.482482 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.484383 (+ 1901us) server_negotiation.cc:407] Connection header received
0806 03:56:05.484476 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.484477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.484488 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.484516 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.484864 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43498: BlockingRecv error: recv got EOF from 127.0.0.1:43498 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260806 03:56:05.485740 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.483067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58268 (local address 127.12.15.125:46045)
0806 03:56:05.483109 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.483111 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.485159 (+ 2048us) server_negotiation.cc:407] Connection header received
0806 03:56:05.485336 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.485337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.485348 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.485378 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.485680 (+ 302us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58268: BlockingRecv error: recv got EOF from 127.0.0.1:58268 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260806 03:56:05.491231 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.490451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.12.15.125:46045)
0806 03:56:05.490488 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.490489 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.490691 (+ 202us) server_negotiation.cc:407] Connection header received
0806 03:56:05.490795 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.490796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.490817 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.490847 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.491158 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58280: BlockingRecv error: recv got EOF from 127.0.0.1:58280 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 03:56:05.494284 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.493488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58296 (local address 127.12.15.125:46045)
0806 03:56:05.493523 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.493524 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.493769 (+ 245us) server_negotiation.cc:407] Connection header received
0806 03:56:05.493856 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.493857 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.493867 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.493895 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.494237 (+ 342us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58296: BlockingRecv error: recv got EOF from 127.0.0.1:58296 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 03:56:05.498198 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.497373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58306 (local address 127.12.15.125:46045)
0806 03:56:05.497407 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.497409 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.497657 (+ 248us) server_negotiation.cc:407] Connection header received
0806 03:56:05.497750 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.497751 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.497761 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.497788 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.498099 (+ 311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58306: BlockingRecv error: recv got EOF from 127.0.0.1:58306 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 03:56:05.507176 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.506402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58314 (local address 127.12.15.125:46045)
0806 03:56:05.506437 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.506438 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.506662 (+ 224us) server_negotiation.cc:407] Connection header received
0806 03:56:05.506757 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.506758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.506768 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.506796 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.507132 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58314: BlockingRecv error: recv got EOF from 127.0.0.1:58314 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 03:56:05.517132 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.516310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58330 (local address 127.12.15.125:46045)
0806 03:56:05.516347 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.516349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.516606 (+ 257us) server_negotiation.cc:407] Connection header received
0806 03:56:05.516700 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.516701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.516711 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.516739 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.517084 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58330: BlockingRecv error: recv got EOF from 127.0.0.1:58330 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 03:56:05.535449 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.534533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.12.15.125:46045)
0806 03:56:05.534573 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.534575 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.534859 (+ 284us) server_negotiation.cc:407] Connection header received
0806 03:56:05.534963 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.534964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.534975 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.535004 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.535392 (+ 388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58336: BlockingRecv error: recv got EOF from 127.0.0.1:58336 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 03:56:05.569448 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.568671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58350 (local address 127.12.15.125:46045)
0806 03:56:05.568724 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.568726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.568855 (+ 129us) server_negotiation.cc:407] Connection header received
0806 03:56:05.568923 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.568925 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.568936 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.569019 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.569384 (+ 365us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58350: BlockingRecv error: recv got EOF from 127.0.0.1:58350 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260806 03:56:05.635397 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.634674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58366 (local address 127.12.15.125:46045)
0806 03:56:05.634747 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.634749 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.634830 (+ 81us) server_negotiation.cc:407] Connection header received
0806 03:56:05.634912 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.634913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.634924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.634972 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.635334 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58366: BlockingRecv error: recv got EOF from 127.0.0.1:58366 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
W20260806 03:56:05.765154 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:05.764503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.12.15.125:46045)
0806 03:56:05.764590 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:05.764592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:05.764658 (+ 66us) server_negotiation.cc:407] Connection header received
0806 03:56:05.764713 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:05.764714 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:05.764725 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:05.764777 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:05.765089 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38372: BlockingRecv error: recv got EOF from 127.0.0.1:38372 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260806 03:56:06.022915 14422 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:06.022385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38386 (local address 127.12.15.125:46045)
0806 03:56:06.022438 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:06.022440 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:06.022450 (+ 10us) server_negotiation.cc:407] Connection header received
0806 03:56:06.022494 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:06.022495 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:06.022505 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:06.022555 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:06.022863 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38386: BlockingRecv error: recv got EOF from 127.0.0.1:38386 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:56:06.371596 14565 catalog_manager.cc:1259] Loaded cluster ID: 952c39e68d9f48d8b79a3d059548baf3
I20260806 03:56:06.371673 14565 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8: loading cluster ID for follower catalog manager: success
I20260806 03:56:06.372478 14565 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8: acquiring CA information for follower catalog manager: success
I20260806 03:56:06.372782 14565 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b84789f4e593461191cbca3f5ca17cc8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:56:06.436182 14631 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6b605391bde0422d8d71a3093a71896e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:56:06.538396 14744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:06.536069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38396 (local address 127.12.15.125:46045)
0806 03:56:06.536207 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:06.536209 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:06.536222 (+ 13us) server_negotiation.cc:407] Connection header received
0806 03:56:06.536292 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:06.536293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:06.536346 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:06.536418 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:06.536537 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.537037 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.537738 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.537916 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.537959 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:06.538091 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:06.538220 (+ 129us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:06.538273 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:06.538291 (+ 18us) server_negotiation.cc:299] Negotiation successful
0806 03:56:06.538308 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":70,"threads_started":1}
I20260806 03:56:06.542191 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:06.540167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58712 (local address 127.12.15.124:35871)
0806 03:56:06.540278 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:06.540280 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:06.540288 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:56:06.540326 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:06.540327 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:06.540363 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:06.540426 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:06.540503 (+ 77us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.540968 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.541597 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.541778 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.541820 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:06.541908 (+ 88us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:06.542004 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:06.542052 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:06.542067 (+ 15us) server_negotiation.cc:299] Negotiation successful
0806 03:56:06.542082 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91,"thread_start_us":58,"threads_started":1}
I20260806 03:56:06.546653 14744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:06.543849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38404 (local address 127.12.15.125:46045)
0806 03:56:06.543887 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:06.543889 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:06.544135 (+ 246us) server_negotiation.cc:407] Connection header received
0806 03:56:06.544186 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:06.544187 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:06.544195 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:06.544251 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:06.544407 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.545121 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.545874 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.546187 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.546240 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:06.546409 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:06.546436 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:06.546462 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:06.546557 (+ 95us) server_negotiation.cc:299] Negotiation successful
0806 03:56:06.546582 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:56:06.546695 14761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:06.543683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42464 (local address 127.12.15.126:36399)
0806 03:56:06.543824 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:06.543826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:06.543987 (+ 161us) server_negotiation.cc:407] Connection header received
0806 03:56:06.544084 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:06.544085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:06.544137 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:06.544216 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:06.544322 (+ 106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.544837 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.545812 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.546057 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.546117 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:06.546333 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:06.546439 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:06.546476 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:06.546584 (+ 108us) server_negotiation.cc:299] Negotiation successful
0806 03:56:06.546608 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":76,"threads_started":1}
I20260806 03:56:06.546764 14754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:06.543810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58722 (local address 127.12.15.124:35871)
0806 03:56:06.543849 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:06.543851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:06.543981 (+ 130us) server_negotiation.cc:407] Connection header received
0806 03:56:06.544007 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:06.544008 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:06.544018 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:06.544081 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:06.544362 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.545072 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.546048 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:06.546326 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:06.546384 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:06.546520 (+ 136us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:06.546551 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:06.546588 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:06.546673 (+ 85us) server_negotiation.cc:299] Negotiation successful
0806 03:56:06.546704 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260806 03:56:06.548476 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14197
I20260806 03:56:06.568491 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14304
I20260806 03:56:06.575152 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14427
2026-08-06T03:56:06Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2939 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 06 03:56:06 dist-test-slave-bqcl krb5kdc[14770](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 06 03:56:06 dist-test-slave-bqcl krb5kdc[14770](info): set up 2 sockets
Aug 06 03:56:06 dist-test-slave-bqcl krb5kdc[14770](info): commencing operation
krb5kdc: starting...
W20260806 03:56:07.711689 12349 mini_kdc.cc:121] Time spent starting KDC: real 1.109s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/test-user.keytab.
Aug 06 03:56:07 dist-test-slave-bqcl krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-06T03:56:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-06T03:56:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.12.15.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
Entry for principal kudu/127.12.15.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab.
I20260806 03:56:07.832763 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:40467
--webserver_interface=127.12.15.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:40467,127.12.15.125:35519,127.12.15.124:46253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260806 03:56:07.914007 14791 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:56:07.914225 14791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:07.914242 14791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:07.914259 14791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:07.915318 14791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:07.915356 14791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:07.915369 14791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:07.915380 14791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:07.915391 14791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:07.916793 14791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:40467,127.12.15.125:35519,127.12.15.124:46253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.126.keytab
--principal=kudu/127.12.15.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.126:40467
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.12.15.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:07.916978 14791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:07.917142 14791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:56:07.921816 14799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:07.921789 14798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:07.921824 14791 server_base.cc:1034] running on GCE node
W20260806 03:56:07.921780 14801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:07.922237 14791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:07.922530 14791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:07.923668 14791 hybrid_clock.cc:648] HybridClock initialized: now 1785988567923656 us; error 31 us; skew 500 ppm
Aug 06 03:56:07 dist-test-slave-bqcl krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988567, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:07.929484 14791 init.cc:377] Logged in from keytab as kudu/127.12.15.126@KRBTEST.COM (short username kudu)
I20260806 03:56:07.929953 14791 webserver.cc:460] Webserver started at http://127.12.15.126:36761/ using document root <none> and password file <none>
I20260806 03:56:07.930213 14791 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:07.930274 14791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:07.930402 14791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:07.931445 14791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/data/instance:
uuid: "cabfa35c86e7474eb4f9c63b22838e54"
format_stamp: "Formatted at 2026-08-06 03:56:07 on dist-test-slave-bqcl"
server_key: "1b4e9d60e16858c79db040f59ba184a4"
server_key_iv: "7297d3097629b94b38b3c22353395f0e"
server_key_version: "encryptionkey@0"
I20260806 03:56:07.931787 14791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/wal/instance:
uuid: "cabfa35c86e7474eb4f9c63b22838e54"
format_stamp: "Formatted at 2026-08-06 03:56:07 on dist-test-slave-bqcl"
server_key: "1b4e9d60e16858c79db040f59ba184a4"
server_key_iv: "7297d3097629b94b38b3c22353395f0e"
server_key_version: "encryptionkey@0"
I20260806 03:56:07.933106 14791 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260806 03:56:07.933807 14810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:07.933967 14791 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:56:07.934053 14791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/wal
uuid: "cabfa35c86e7474eb4f9c63b22838e54"
format_stamp: "Formatted at 2026-08-06 03:56:07 on dist-test-slave-bqcl"
server_key: "1b4e9d60e16858c79db040f59ba184a4"
server_key_iv: "7297d3097629b94b38b3c22353395f0e"
server_key_version: "encryptionkey@0"
I20260806 03:56:07.934116 14791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:07.944500 14791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:07.946328 14791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:07.946476 14791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:07.950451 14791 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.126:40467
I20260806 03:56:07.950503 14892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.126:40467 every 8 connection(s)
I20260806 03:56:07.950819 14791 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/data/info.pb
I20260806 03:56:07.951833 14893 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:07.953377 14893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:07.957398 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14791
I20260806 03:56:07.957487 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-0/wal/instance
I20260806 03:56:07.957661 12349 external_mini_cluster.cc:1428] Setting key 3164b74acb4272edb79a6adfb18bae8e
Aug 06 03:56:07 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988567, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:56:07.962453 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:07.954160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.12.15.126:40467)
0806 03:56:07.954397 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:07.954400 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:07.954441 (+ 41us) server_negotiation.cc:407] Connection header received
0806 03:56:07.954954 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:07.954956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:07.954972 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:07.955045 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:07.955546 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:07.956330 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:07.957171 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:07.957499 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:07.957601 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:07.959576 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:07.959582 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:07.959583 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:07.959589 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:07.960989 (+ 1400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:07.961337 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:07.961341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:07.961341 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:07.961417 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:07.961608 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:07.961609 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:07.961609 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:07.961819 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:07.961890 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:07.962075 (+ 185us) server_negotiation.cc:299] Negotiation successful
0806 03:56:07.962213 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"spinlock_wait_cycles":8960,"thread_start_us":108,"threads_started":1}
I20260806 03:56:07.962453 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:07.954107 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40467 (local address 127.0.0.1:43016)
0806 03:56:07.954337 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:07.954372 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:07.954834 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:07.955064 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:07.955066 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:07.955307 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:07.955461 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:07.955464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:07.956413 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:07.956414 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:07.957110 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:07.957112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:07.957596 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:07.957597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:07.957666 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:07.958167 (+ 501us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:07.958176 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:07.959498 (+ 1322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:07.961066 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:07.961073 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:07.961075 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:07.961269 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:07.961474 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:07.961475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:07.961476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:07.961556 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:07.961881 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:07.961887 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:07.961951 (+ 64us) client_negotiation.cc:769] Sending connection context
0806 03:56:07.962030 (+ 79us) client_negotiation.cc:240] Negotiation successful
0806 03:56:07.962293 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"mutex_wait_us":121,"thread_start_us":106,"threads_started":1}
I20260806 03:56:07.963145 14893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:07.963464 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:07.963355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34446)
0806 03:56:07.963394 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:07.963440 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260806 03:56:07.963580 14812 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
I20260806 03:56:07.963963 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:07.963875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34452)
0806 03:56:07.963915 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:07.963939 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 03:56:07.964053 14893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:35519: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.12.15.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab.
Entry for principal kudu/127.12.15.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab.
I20260806 03:56:07.982276 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.125:35519
--webserver_interface=127.12.15.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab
--principal=kudu/127.12.15.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:40467,127.12.15.125:35519,127.12.15.124:46253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:56:08.013265 14893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } attempt: 1
I20260806 03:56:08.013813 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.013676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34456)
0806 03:56:08.013739 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.013782 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260806 03:56:08.014189 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.014082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34470)
0806 03:56:08.014135 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.014164 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260806 03:56:08.014304 14893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:35519: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
W20260806 03:56:08.065987 14905 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:56:08.066236 14905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:08.066264 14905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:08.066289 14905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:08.067441 14905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:08.067479 14905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:08.067497 14905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:08.067512 14905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:08.067528 14905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:08.068960 14905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:40467,127.12.15.125:35519,127.12.15.124:46253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.125.keytab
--principal=kudu/127.12.15.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.125:35519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.12.15.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:08.069226 14905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:08.069434 14905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:56:08.074141 14913 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:08.074164 14918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:08.074270 14905 server_base.cc:1034] running on GCE node
W20260806 03:56:08.074354 14914 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:08.074602 14905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:08.074883 14905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:08.076020 14905 hybrid_clock.cc:648] HybridClock initialized: now 1785988568076006 us; error 30 us; skew 500 ppm
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988568, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:08.078678 14905 init.cc:377] Logged in from keytab as kudu/127.12.15.125@KRBTEST.COM (short username kudu)
I20260806 03:56:08.079058 14905 webserver.cc:460] Webserver started at http://127.12.15.125:39089/ using document root <none> and password file <none>
I20260806 03:56:08.079231 14905 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:08.079272 14905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:08.079363 14905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:08.080334 14905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/data/instance:
uuid: "64467c904dc34afc936202ac2cc90a0a"
format_stamp: "Formatted at 2026-08-06 03:56:08 on dist-test-slave-bqcl"
server_key: "1729afa1670dff427075ef8217ddfec8"
server_key_iv: "22ccb1babcada9020b0f0397928b0731"
server_key_version: "encryptionkey@0"
I20260806 03:56:08.080627 14905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/wal/instance:
uuid: "64467c904dc34afc936202ac2cc90a0a"
format_stamp: "Formatted at 2026-08-06 03:56:08 on dist-test-slave-bqcl"
server_key: "1729afa1670dff427075ef8217ddfec8"
server_key_iv: "22ccb1babcada9020b0f0397928b0731"
server_key_version: "encryptionkey@0"
I20260806 03:56:08.081794 14905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260806 03:56:08.082449 14893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } attempt: 2
I20260806 03:56:08.082492 14928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:08.082685 14905 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:08.082783 14905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/wal
uuid: "64467c904dc34afc936202ac2cc90a0a"
format_stamp: "Formatted at 2026-08-06 03:56:08 on dist-test-slave-bqcl"
server_key: "1729afa1670dff427075ef8217ddfec8"
server_key_iv: "22ccb1babcada9020b0f0397928b0731"
server_key_version: "encryptionkey@0"
I20260806 03:56:08.082827 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.082695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34484)
0806 03:56:08.082745 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.082788 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260806 03:56:08.082855 14905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:08.083182 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.083082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34492)
0806 03:56:08.083127 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.083154 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260806 03:56:08.083282 14893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.125:35519: Network error: Client connection negotiation failed: client connection to 127.12.15.125:35519: connect: Connection refused (error 111)
I20260806 03:56:08.092053 14905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:08.094041 14905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:08.094193 14905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:08.097970 14905 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.125:35519
I20260806 03:56:08.098008 15003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.125:35519 every 8 connection(s)
I20260806 03:56:08.098367 14905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/data/info.pb
I20260806 03:56:08.099331 15004 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:08.100914 15004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988568, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:56:08.107573 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 14905
I20260806 03:56:08.107666 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-1/wal/instance
I20260806 03:56:08.107839 12349 external_mini_cluster.cc:1428] Setting key 3d03858b4d27d5685a5fc5a83df7d4e2
I20260806 03:56:08.109342 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.101633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43028 (local address 127.12.15.126:40467)
0806 03:56:08.101730 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.101732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.101947 (+ 215us) server_negotiation.cc:407] Connection header received
0806 03:56:08.102425 (+ 478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.102426 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.102438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.102498 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.103073 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.103546 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.104460 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.104668 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.104720 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.106752 (+ 2032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.106753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.106753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.106761 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.108089 (+ 1328us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.108506 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.108507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.108507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.108558 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.108790 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.108791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.108791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.108887 (+ 96us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.108931 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.109214 (+ 283us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.109277 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260806 03:56:08.109587 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.101551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40467 (local address 127.0.0.1:43028)
0806 03:56:08.101775 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.101817 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:08.102333 (+ 516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:08.102562 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:08.102566 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:08.102800 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:08.102990 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.102994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.103611 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.103611 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.104388 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.104390 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.104750 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.104751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.104843 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.105303 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:08.105312 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:08.106672 (+ 1360us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:08.108163 (+ 1491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.108171 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.108173 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.108437 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.108609 (+ 172us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.108610 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.108610 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.108737 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.108936 (+ 199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:08.108941 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:08.109095 (+ 154us) client_negotiation.cc:769] Sending connection context
0806 03:56:08.109204 (+ 109us) client_negotiation.cc:240] Negotiation successful
0806 03:56:08.109360 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":134,"threads_started":1}
I20260806 03:56:08.110180 15004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988568, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
WARNING: no policy specified for kudu/127.12.15.124@KRBTEST.COM; defaulting to no policy
I20260806 03:56:08.117391 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.110431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34508)
0806 03:56:08.110479 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.110490 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:08.110535 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:08.110764 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:08.110765 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:08.110902 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:08.110962 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.110964 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.111790 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.111791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.112523 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.112527 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.112996 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.112997 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.113071 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.113401 (+ 330us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:08.113406 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:08.114796 (+ 1390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:08.116361 (+ 1565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.116362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.116362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.116556 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.116824 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.116825 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.116825 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.116890 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.117145 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:08.117146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:08.117201 (+ 55us) client_negotiation.cc:769] Sending connection context
0806 03:56:08.117249 (+ 48us) client_negotiation.cc:240] Negotiation successful
0806 03:56:08.117327 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260806 03:56:08.117499 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.110496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34508 (local address 127.12.15.125:35519)
0806 03:56:08.110637 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.110641 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.110656 (+ 15us) server_negotiation.cc:407] Connection header received
0806 03:56:08.110672 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.110674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.110684 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.110744 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.111030 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.111711 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.112609 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.112925 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.112975 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.114882 (+ 1907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.114890 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.114890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.114896 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.116278 (+ 1382us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.116685 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.116689 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.116689 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.116754 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.116949 (+ 195us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.116949 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.116950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.117093 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.117158 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.117291 (+ 133us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.117386 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":118,"thread_start_us":86,"threads_started":1}
I20260806 03:56:08.118182 15004 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:08.118496 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.118394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56510)
0806 03:56:08.118432 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.118476 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260806 03:56:08.118693 14935 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.12.15.124@KRBTEST.COM" created.
I20260806 03:56:08.119496 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.119025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56518)
0806 03:56:08.119437 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.119472 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":394}
W20260806 03:56:08.119592 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:46253: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.12.15.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab.
Entry for principal kudu/127.12.15.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab.
I20260806 03:56:08.133620 12349 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskGVWHtl/build/release/bin/kudu
/tmp/dist-test-taskGVWHtl/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.124:46253
--webserver_interface=127.12.15.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab
--principal=kudu/127.12.15.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.12.15.84:36501
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.12.15.126:40467,127.12.15.125:35519,127.12.15.124:46253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260806 03:56:08.168828 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } attempt: 1
I20260806 03:56:08.169301 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.169178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56530)
0806 03:56:08.169234 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.169269 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:56:08.169607 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.169525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56534)
0806 03:56:08.169563 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.169587 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260806 03:56:08.169690 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:46253: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
I20260806 03:56:08.174414 14893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } attempt: 3
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988567, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:56:08.181602 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.174695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34524)
0806 03:56:08.174744 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.174754 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:08.174795 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:08.174949 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:08.174950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:08.175121 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:08.175198 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.175201 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.175847 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.175848 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.176795 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.176797 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.177170 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.177171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.177242 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.177756 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:08.177761 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:08.179243 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:08.180670 (+ 1427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.180672 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.180672 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.180833 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.181004 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.181004 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.181005 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.181061 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.181284 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:08.181285 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:08.181335 (+ 50us) client_negotiation.cc:769] Sending connection context
0806 03:56:08.181395 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:56:08.181502 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 03:56:08.181600 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.174786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.12.15.125:35519)
0806 03:56:08.174849 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.174851 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.174860 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:56:08.174882 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.174882 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.174893 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.174937 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.175275 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.175771 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.176879 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.177096 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.177154 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.179340 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.179341 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.179341 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.179348 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.180586 (+ 1238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.180896 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.180897 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.180897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.180949 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.181111 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.181112 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.181112 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.181228 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.181285 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.181424 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.181502 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260806 03:56:08.182087 14893 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:08.182454 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.182366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56550)
0806 03:56:08.182402 (+ 36us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.182432 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260806 03:56:08.182781 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.182698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56566)
0806 03:56:08.182732 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.182755 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
W20260806 03:56:08.182865 14893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:46253: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
I20260806 03:56:08.214011 14893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } attempt: 1
I20260806 03:56:08.214452 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.214337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56574)
0806 03:56:08.214390 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.214422 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260806 03:56:08.214862 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.214688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56578)
0806 03:56:08.214806 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.214831 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":77,"threads_started":1}
W20260806 03:56:08.214972 14893 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:46253: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
W20260806 03:56:08.216281 15016 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260806 03:56:08.216483 15016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260806 03:56:08.216508 15016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260806 03:56:08.216549 15016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260806 03:56:08.217644 15016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260806 03:56:08.217684 15016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260806 03:56:08.217702 15016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260806 03:56:08.217718 15016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260806 03:56:08.217743 15016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260806 03:56:08.219259 15016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.12.15.84:36501
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.12.15.126:40467,127.12.15.125:35519,127.12.15.124:46253
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/krb5kdc/kudu_127.12.15.124.keytab
--principal=kudu/127.12.15.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.12.15.124:46253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.12.15.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 06 Aug 2026 02:39:18 UTC on ccycloud.quasar-kgtzdm.root.comops.site
build id 4569539
I20260806 03:56:08.219507 15016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260806 03:56:08.219725 15016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260806 03:56:08.224480 15030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260806 03:56:08.224474 15034 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:08.224490 15016 server_base.cc:1034] running on GCE node
W20260806 03:56:08.224471 15031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260806 03:56:08.224967 15016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260806 03:56:08.225272 15016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260806 03:56:08.226413 15016 hybrid_clock.cc:648] HybridClock initialized: now 1785988568226400 us; error 30 us; skew 500 ppm
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988568, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260806 03:56:08.229051 15016 init.cc:377] Logged in from keytab as kudu/127.12.15.124@KRBTEST.COM (short username kudu)
I20260806 03:56:08.229509 15016 webserver.cc:460] Webserver started at http://127.12.15.124:41353/ using document root <none> and password file <none>
I20260806 03:56:08.229712 15016 fs_manager.cc:359] Metadata directory not provided
I20260806 03:56:08.229764 15016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260806 03:56:08.229899 15016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260806 03:56:08.231029 15016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/data/instance:
uuid: "b0ac14306acd45aeafd79b6c13f91eb5"
format_stamp: "Formatted at 2026-08-06 03:56:08 on dist-test-slave-bqcl"
server_key: "aa4347f67983d9c6d247747a1be4d6be"
server_key_iv: "79fda754f2829fae6a5ff2384f2ef248"
server_key_version: "encryptionkey@0"
I20260806 03:56:08.231387 15016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/wal/instance:
uuid: "b0ac14306acd45aeafd79b6c13f91eb5"
format_stamp: "Formatted at 2026-08-06 03:56:08 on dist-test-slave-bqcl"
server_key: "aa4347f67983d9c6d247747a1be4d6be"
server_key_iv: "79fda754f2829fae6a5ff2384f2ef248"
server_key_version: "encryptionkey@0"
I20260806 03:56:08.232724 15016 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260806 03:56:08.233435 15045 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260806 03:56:08.233594 15016 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260806 03:56:08.233680 15016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/data,/tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/wal
uuid: "b0ac14306acd45aeafd79b6c13f91eb5"
format_stamp: "Formatted at 2026-08-06 03:56:08 on dist-test-slave-bqcl"
server_key: "aa4347f67983d9c6d247747a1be4d6be"
server_key_iv: "79fda754f2829fae6a5ff2384f2ef248"
server_key_version: "encryptionkey@0"
I20260806 03:56:08.233750 15016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260806 03:56:08.237814 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } attempt: 2
I20260806 03:56:08.238230 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.238085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56584)
0806 03:56:08.238162 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.238199 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260806 03:56:08.238566 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.238460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56596)
0806 03:56:08.238487 (+ 27us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.238514 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260806 03:56:08.238647 15004 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.12.15.124:46253: Network error: Client connection negotiation failed: client connection to 127.12.15.124:46253: connect: Connection refused (error 111)
I20260806 03:56:08.250615 15016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260806 03:56:08.252594 15016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260806 03:56:08.252705 15016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260806 03:56:08.256568 15016 rpc_server.cc:307] RPC server started. Bound to: 127.12.15.124:46253
I20260806 03:56:08.256620 15130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.12.15.124:46253 every 8 connection(s)
I20260806 03:56:08.256985 15016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/data/info.pb
I20260806 03:56:08.258005 15131 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260806 03:56:08.258648 12349 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskGVWHtl/build/release/bin/kudu as pid 15016
I20260806 03:56:08.258754 12349 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskGVWHtl/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1785988295328500-12349-0/minicluster-data/master-2/wal/instance
I20260806 03:56:08.258936 12349 external_mini_cluster.cc:1428] Setting key 80696ddc53a9f3ecf86d5e5031cefc94
I20260806 03:56:08.259932 15131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988568, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
I20260806 03:56:08.268429 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.260806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43042 (local address 127.12.15.126:40467)
0806 03:56:08.260859 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.260861 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.260982 (+ 121us) server_negotiation.cc:407] Connection header received
0806 03:56:08.261037 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.261038 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.261048 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.261093 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.261631 (+ 538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.262079 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.262952 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.263137 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.263188 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.265454 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.265456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.265456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.265463 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.266981 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.267389 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.267391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.267391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.267458 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.267740 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.267741 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.267741 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.267878 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.267941 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.268247 (+ 306us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.268334 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260806 03:56:08.268631 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.259430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56608 (local address 127.12.15.124:46253)
0806 03:56:08.259632 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.259643 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.259677 (+ 34us) server_negotiation.cc:407] Connection header received
0806 03:56:08.260167 (+ 490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.260176 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.260241 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.260390 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.261035 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.262155 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.262887 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.263252 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.263321 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.265893 (+ 2572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.265900 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.265901 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.265907 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.267462 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.267813 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.267816 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.267817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.267871 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.268055 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.268055 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.268055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.268185 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.268326 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.268408 (+ 82us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.268477 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":88,"threads_started":1}
I20260806 03:56:08.268812 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.260739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.126:40467 (local address 127.0.0.1:43042)
0806 03:56:08.260878 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.260916 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:08.260982 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:08.261119 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:08.261122 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:08.261442 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:08.261554 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.261560 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.262176 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.262177 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.262861 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.262864 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.263218 (+ 354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.263219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.263316 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.263929 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:08.263938 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:08.265357 (+ 1419us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:08.267062 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.267068 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.267070 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.267320 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.267547 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.267547 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.267548 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.267677 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.267936 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:08.267942 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:08.268109 (+ 167us) client_negotiation.cc:769] Sending connection context
0806 03:56:08.268189 (+ 80us) client_negotiation.cc:240] Negotiation successful
0806 03:56:08.268335 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260806 03:56:08.269438 15131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } has no permanent_uuid. Determining permanent_uuid...
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988568, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.126@KRBTEST.COM
I20260806 03:56:08.275516 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.269668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.125:35519 (local address 127.0.0.1:34526)
0806 03:56:08.269707 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.269724 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:08.269753 (+ 29us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:08.269861 (+ 108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:08.269861 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:08.269990 (+ 129us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:08.270047 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.270048 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.270676 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.270677 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.271153 (+ 476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.271154 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.271462 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.271462 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.271527 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.271859 (+ 332us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:08.271863 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:08.273019 (+ 1156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:08.274642 (+ 1623us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.274643 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.274643 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.274796 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.274980 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.274981 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.274981 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.275034 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.275246 (+ 212us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:08.275247 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:08.275289 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 03:56:08.275343 (+ 54us) client_negotiation.cc:240] Negotiation successful
0806 03:56:08.275427 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":27}
I20260806 03:56:08.275537 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.269734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34526 (local address 127.12.15.125:35519)
0806 03:56:08.269774 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.269776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.269783 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:56:08.269806 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.269806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.269815 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.269855 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.270112 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.270606 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.271222 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.271408 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.271460 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.273105 (+ 1645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.273106 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.273107 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.273113 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.274571 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.274862 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.274864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.274864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.274923 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.275091 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.275092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.275092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.275191 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.275252 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.275383 (+ 131us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.275450 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260806 03:56:08.275826 15131 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } has no permanent_uuid. Determining permanent_uuid...
I20260806 03:56:08.276160 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.269394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43052 (local address 127.12.15.126:40467)
0806 03:56:08.269426 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.269428 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.269633 (+ 205us) server_negotiation.cc:407] Connection header received
0806 03:56:08.269678 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.269679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.269688 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.269739 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.270339 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.270867 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.271500 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.271692 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.271751 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.273679 (+ 1928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.273680 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.273680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.273688 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.274959 (+ 1271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.275262 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.275263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.275263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.275315 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.275493 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.275493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.275494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.275611 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.275792 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.275997 (+ 205us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.276075 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14}
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988568, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.124@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988567, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.12.15.125@KRBTEST.COM
I20260806 03:56:08.281873 15136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.276065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56610)
0806 03:56:08.276105 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.276115 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:08.276148 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:08.276286 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:08.276287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:08.276423 (+ 136us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:08.276450 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.276452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.277275 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.277276 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.277728 (+ 452us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.277730 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.278064 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.278064 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.278109 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.278485 (+ 376us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:08.278488 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:08.279505 (+ 1017us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:08.280905 (+ 1400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.280906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.280906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.281059 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.281239 (+ 180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.281240 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.281240 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.281297 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.281537 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:08.281538 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:08.281587 (+ 49us) client_negotiation.cc:769] Sending connection context
0806 03:56:08.281654 (+ 67us) client_negotiation.cc:240] Negotiation successful
0806 03:56:08.281746 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26}
I20260806 03:56:08.281883 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.276109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56610 (local address 127.12.15.124:46253)
0806 03:56:08.276140 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.276142 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.276149 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:56:08.276187 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.276187 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.276196 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.276255 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.276524 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.277208 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.277803 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.278008 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.278061 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.279576 (+ 1515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.279577 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.279577 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.279583 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.280840 (+ 1257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.281126 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.281128 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.281128 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.281186 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.281361 (+ 175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.281361 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.281362 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.281480 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.281541 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.281680 (+ 139us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.281795 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":17}
I20260806 03:56:08.283432 15131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5: Bootstrap starting.
I20260806 03:56:08.283689 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.276951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.12.15.125:35519)
0806 03:56:08.276983 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.276985 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.276993 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:56:08.277029 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.277030 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.277038 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.277078 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.277534 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.277958 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.278741 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.278938 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.278990 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.281163 (+ 2173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.281164 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.281165 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.281171 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.282756 (+ 1585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.283033 (+ 277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.283034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.283034 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.283087 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.283246 (+ 159us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.283247 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.283247 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.283348 (+ 101us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.283491 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.283526 (+ 35us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.283601 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260806 03:56:08.284394 15131 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:08.284736 15131 log.cc:824] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:08.285100 12349 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260806 03:56:08.285596 15131 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5: No bootstrap required, opened a new log
I20260806 03:56:08.286020 14848 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "cabfa35c86e7474eb4f9c63b22838e54"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43052
I20260806 03:56:08.287248 15131 raft_consensus.cc:348] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } }
I20260806 03:56:08.287371 15131 raft_consensus.cc:729] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b0ac14306acd45aeafd79b6c13f91eb5, State: Initialized, Role: FOLLOWER
I20260806 03:56:08.287449 15131 consensus_queue.cc:260] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } }
I20260806 03:56:08.287714 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.287784 15131 sys_catalog.cc:564] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:56:08.287810 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [sys.catalog]: This master's current role is: FOLLOWER
W20260806 03:56:08.289415 15159 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:56:08.289470 15159 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:56:08.290092 14893 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } attempt: 2
I20260806 03:56:08.289804 15016 master_runner.cc:186] Error getting master registration for 127.12.15.126:40467: OK, instance_id { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" instance_seqno: 1785988567949308 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:56:08.292616 15016 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988567, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.126@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
I20260806 03:56:08.299635 14895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.290332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56626)
0806 03:56:08.292700 (+ 2368us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.292712 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:08.292759 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:08.292942 (+ 183us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:08.292944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:08.293112 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:08.293158 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.293159 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.294000 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.294000 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.294813 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.294814 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.295266 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.295267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.295319 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.295697 (+ 378us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:08.295702 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:08.296804 (+ 1102us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:08.298577 (+ 1773us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.298578 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.298579 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.298787 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.298991 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.298991 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.298991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.299047 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.299342 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:08.299343 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:08.299386 (+ 43us) client_negotiation.cc:769] Sending connection context
0806 03:56:08.299446 (+ 60us) client_negotiation.cc:240] Negotiation successful
0806 03:56:08.299535 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2345}
I20260806 03:56:08.299641 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.290382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56626 (local address 127.12.15.124:46253)
0806 03:56:08.292826 (+ 2444us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.292828 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.292835 (+ 7us) server_negotiation.cc:407] Connection header received
0806 03:56:08.292852 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.292853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.292863 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.292906 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.293236 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.293929 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.294885 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.295179 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.295268 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.296878 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.296879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.296880 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.296887 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.298500 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.298860 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.298862 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.298863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.298931 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.299107 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.299108 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.299108 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.299277 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.299349 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.299481 (+ 132us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.299556 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2426}
I20260806 03:56:08.301151 14893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54: Bootstrap starting.
I20260806 03:56:08.301841 14893 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:08.302153 14893 log.cc:824] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:08.302845 14893 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54: No bootstrap required, opened a new log
I20260806 03:56:08.303833 14893 raft_consensus.cc:348] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } }
I20260806 03:56:08.303946 14893 raft_consensus.cc:729] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cabfa35c86e7474eb4f9c63b22838e54, State: Initialized, Role: FOLLOWER
I20260806 03:56:08.304021 14893 consensus_queue.cc:260] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } }
I20260806 03:56:08.304286 15163 sys_catalog.cc:455] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.304338 14893 sys_catalog.cc:564] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: configured and running, proceeding with master startup.
I20260806 03:56:08.304400 14848 raft_consensus.cc:482] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260806 03:56:08.304495 14848 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:56:08.305341 14848 raft_consensus.cc:504] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } }
W20260806 03:56:08.306483 15182 catalog_manager.cc:1558] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:56:08.306577 15182 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:56:08.307026 15091 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cabfa35c86e7474eb4f9c63b22838e54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b0ac14306acd45aeafd79b6c13f91eb5"
I20260806 03:56:08.307169 15091 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:56:08.308091 15091 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cabfa35c86e7474eb4f9c63b22838e54 in term 1.
I20260806 03:56:08.308365 14816 leader_election.cc:302] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b0ac14306acd45aeafd79b6c13f91eb5, cabfa35c86e7474eb4f9c63b22838e54; no voters:
I20260806 03:56:08.310190 14848 leader_election.cc:288] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [CANDIDATE]: Term 1 election: Requested vote from peers 64467c904dc34afc936202ac2cc90a0a (127.12.15.125:35519), b0ac14306acd45aeafd79b6c13f91eb5 (127.12.15.124:46253)
I20260806 03:56:08.304401 15163 sys_catalog.cc:458] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:08.310573 15163 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [term 1 FOLLOWER]: Leader election won for term 1
I20260806 03:56:08.310722 15163 raft_consensus.cc:686] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [term 1 LEADER]: Becoming Leader. State: Replica: cabfa35c86e7474eb4f9c63b22838e54, State: Running, Role: LEADER
I20260806 03:56:08.308447 14791 master_runner.cc:186] Error getting master registration for 127.12.15.125:35519: OK, instance_id { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" instance_seqno: 1785988568096939 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260806 03:56:08.310911 14791 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260806 03:56:08.311031 15163 consensus_queue.cc:237] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } }
I20260806 03:56:08.311892 15163 sys_catalog.cc:455] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cabfa35c86e7474eb4f9c63b22838e54. Latest consensus state: current_term: 1 leader_uuid: "cabfa35c86e7474eb4f9c63b22838e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.312104 15163 sys_catalog.cc:458] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:08.312500 15197 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260806 03:56:08.313459 15197 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260806 03:56:08.322176 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.312022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56634 (local address 127.12.15.124:46253)
0806 03:56:08.312061 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.312062 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.313003 (+ 941us) server_negotiation.cc:407] Connection header received
0806 03:56:08.313097 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.313098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.313109 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.313137 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.313923 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.314697 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.315777 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.316078 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.316160 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.318695 (+ 2535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.318696 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.318697 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.318704 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.320494 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.321090 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.321091 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.321092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.321166 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.321455 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.321456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.321457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.321605 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.321677 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.321986 (+ 309us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.322063 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260806 03:56:08.323066 15091 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [term 1 FOLLOWER]: Refusing update from remote peer cabfa35c86e7474eb4f9c63b22838e54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:56:08.323390 15163 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:56:08.324719 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cabfa35c86e7474eb4f9c63b22838e54. Latest consensus state: current_term: 1 leader_uuid: "cabfa35c86e7474eb4f9c63b22838e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.324817 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:08.325884 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.312450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43060 (local address 127.12.15.126:40467)
0806 03:56:08.312487 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.312489 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.315869 (+ 3380us) server_negotiation.cc:407] Connection header received
0806 03:56:08.315958 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.315959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.315969 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.315997 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.316712 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.317445 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.319318 (+ 1873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.319628 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.319667 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.322999 (+ 3332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.323000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.323000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.323008 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.324599 (+ 1591us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.325029 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.325030 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.325030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.325087 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.325302 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.325303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.325303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.325432 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.325527 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.325715 (+ 188us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.325792 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:56:08.329773 15004 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } attempt: 3
I20260806 03:56:08.331516 15185 sys_catalog.cc:455] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cabfa35c86e7474eb4f9c63b22838e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.331622 15185 sys_catalog.cc:458] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:08.332247 15141 sys_catalog.cc:455] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cabfa35c86e7474eb4f9c63b22838e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.332340 15141 sys_catalog.cc:458] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5 [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:08.332674 15203 mvcc.cc:205] Tried to move back new op lower bound from 7315409175848681472 to 7315409175803740160. Current Snapshot: MvccSnapshot[applied={T|T < 7315409175848681472}]
Aug 06 03:56:08 dist-test-slave-bqcl krb5kdc[14770](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1785988568, etypes {rep=17 tkt=17 ses=17}, kudu/127.12.15.125@KRBTEST.COM for kudu/127.12.15.124@KRBTEST.COM
I20260806 03:56:08.334978 15010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.312733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34548 (local address 127.12.15.125:35519)
0806 03:56:08.312782 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.312784 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.318544 (+ 5760us) server_negotiation.cc:407] Connection header received
0806 03:56:08.318640 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.318641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.318652 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.318681 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.320953 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.321694 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.325007 (+ 3313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.325775 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.325844 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.327836 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.327837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.327838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.327845 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.329564 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.329997 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.329999 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.329999 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.330059 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.332829 (+ 2770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.332830 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.332830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.332968 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.333046 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.334809 (+ 1763us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.334884 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260806 03:56:08.336163 15197 catalog_manager.cc:1347] Generated new cluster ID: b7b0db7d190c4b48a19b11df03faaeb5
I20260806 03:56:08.336228 15197 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260806 03:56:08.338929 15008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.330039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.12.15.124:46253 (local address 127.0.0.1:56636)
0806 03:56:08.330089 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0806 03:56:08.330098 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0806 03:56:08.330162 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0806 03:56:08.330282 (+ 120us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0806 03:56:08.330284 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0806 03:56:08.330427 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0806 03:56:08.330469 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.330471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.331317 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.331318 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.332097 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0806 03:56:08.332100 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.332593 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.332594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0806 03:56:08.332645 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.333094 (+ 449us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0806 03:56:08.333098 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0806 03:56:08.334545 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0806 03:56:08.337578 (+ 3033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.337579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.337580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.337781 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.338098 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0806 03:56:08.338099 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0806 03:56:08.338099 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0806 03:56:08.338201 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0806 03:56:08.338654 (+ 453us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0806 03:56:08.338655 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0806 03:56:08.338697 (+ 42us) client_negotiation.cc:769] Sending connection context
0806 03:56:08.338742 (+ 45us) client_negotiation.cc:240] Negotiation successful
0806 03:56:08.338832 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260806 03:56:08.339716 15135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:08.330057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56636 (local address 127.12.15.124:46253)
0806 03:56:08.330098 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.330100 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.330176 (+ 76us) server_negotiation.cc:407] Connection header received
0806 03:56:08.330201 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.330202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.330212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.330272 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.330535 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.331213 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.332187 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:08.332487 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:08.332540 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:08.334625 (+ 2085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0806 03:56:08.334626 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0806 03:56:08.334627 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0806 03:56:08.334634 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0806 03:56:08.337415 (+ 2781us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.337914 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.337915 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.337915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.337999 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0806 03:56:08.338289 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0806 03:56:08.338289 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0806 03:56:08.338290 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0806 03:56:08.338533 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0806 03:56:08.339325 (+ 792us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:08.339401 (+ 76us) server_negotiation.cc:299] Negotiation successful
0806 03:56:08.339531 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260806 03:56:08.341485 15004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a: Bootstrap starting.
I20260806 03:56:08.342286 15004 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a: Neither blocks nor log segments found. Creating new log.
I20260806 03:56:08.342620 15004 log.cc:824] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a: Log is configured to *not* fsync() on all Append() calls
I20260806 03:56:08.353312 15004 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a: No bootstrap required, opened a new log
I20260806 03:56:08.356051 15197 catalog_manager.cc:1370] Generated new certificate authority record
I20260806 03:56:08.356365 15004 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } }
I20260806 03:56:08.356629 15004 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64467c904dc34afc936202ac2cc90a0a, State: Initialized, Role: FOLLOWER
I20260806 03:56:08.356814 15197 catalog_manager.cc:1504] Loading token signing keys...
I20260806 03:56:08.356853 15004 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } }
I20260806 03:56:08.357736 14967 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [term 0 FOLLOWER]: Advancing to term 1
I20260806 03:56:08.358712 14967 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [term 1 FOLLOWER]: Refusing update from remote peer cabfa35c86e7474eb4f9c63b22838e54: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260806 03:56:08.358999 15185 consensus_queue.cc:1035] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [LEADER]: Connected to new peer: Peer: permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260806 03:56:08.360477 14969 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "cabfa35c86e7474eb4f9c63b22838e54" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "64467c904dc34afc936202ac2cc90a0a"
I20260806 03:56:08.357631 15004 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [sys.catalog]: configured and running, proceeding with master startup.
W20260806 03:56:08.366369 15241 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260806 03:56:08.366482 15241 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260806 03:56:08.367031 15216 mvcc.cc:205] Tried to move back new op lower bound from 7315409175973502976 to 7315409175803740160. Current Snapshot: MvccSnapshot[applied={T|T < 7315409175848681472}]
I20260806 03:56:08.367844 14969 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate cabfa35c86e7474eb4f9c63b22838e54 for term 1 because replica has last-logged OpId of term: 1 index: 3, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260806 03:56:08.368333 15218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cabfa35c86e7474eb4f9c63b22838e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.368389 15163 sys_catalog.cc:455] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "cabfa35c86e7474eb4f9c63b22838e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.368484 15163 sys_catalog.cc:458] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54 [sys.catalog]: This master's current role is: LEADER
I20260806 03:56:08.368546 15218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:08.368675 15218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [sys.catalog]: SysCatalogTable state changed. Reason: New leader cabfa35c86e7474eb4f9c63b22838e54. Latest consensus state: current_term: 1 leader_uuid: "cabfa35c86e7474eb4f9c63b22838e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.368772 15218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:08.368839 15218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "cabfa35c86e7474eb4f9c63b22838e54" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cabfa35c86e7474eb4f9c63b22838e54" member_type: VOTER last_known_addr { host: "127.12.15.126" port: 40467 } } peers { permanent_uuid: "64467c904dc34afc936202ac2cc90a0a" member_type: VOTER last_known_addr { host: "127.12.15.125" port: 35519 } } peers { permanent_uuid: "b0ac14306acd45aeafd79b6c13f91eb5" member_type: VOTER last_known_addr { host: "127.12.15.124" port: 46253 } } }
I20260806 03:56:08.368922 15218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a [sys.catalog]: This master's current role is: FOLLOWER
I20260806 03:56:08.369732 15197 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cabfa35c86e7474eb4f9c63b22838e54: Generated new TSK 0
I20260806 03:56:08.369902 15197 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260806 03:56:08.396025 15135 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.394423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56638 (local address 127.12.15.124:46253)
0806 03:56:08.394466 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.394468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.395386 (+ 918us) server_negotiation.cc:407] Connection header received
0806 03:56:08.395491 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.395492 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.395502 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.395529 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.395954 (+ 425us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56638: BlockingRecv error: recv got EOF from 127.0.0.1:56638 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 03:56:08.396909 14898 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.394842 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.12.15.126:40467)
0806 03:56:08.394882 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.394883 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.396416 (+ 1533us) server_negotiation.cc:407] Connection header received
0806 03:56:08.396505 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.396506 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.396516 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.396540 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.396849 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43064: BlockingRecv error: recv got EOF from 127.0.0.1:43064 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260806 03:56:08.397778 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.395145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34564 (local address 127.12.15.125:35519)
0806 03:56:08.395182 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.395184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.397234 (+ 2050us) server_negotiation.cc:407] Connection header received
0806 03:56:08.397354 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.397356 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.397366 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.397397 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.397704 (+ 307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34564: BlockingRecv error: recv got EOF from 127.0.0.1:34564 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 03:56:08.400024 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.399254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34570 (local address 127.12.15.125:35519)
0806 03:56:08.399287 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.399288 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.399504 (+ 216us) server_negotiation.cc:407] Connection header received
0806 03:56:08.399624 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.399625 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.399642 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.399672 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.399980 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34570: BlockingRecv error: recv got EOF from 127.0.0.1:34570 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 03:56:08.403163 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.402401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34574 (local address 127.12.15.125:35519)
0806 03:56:08.402434 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.402435 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.402673 (+ 238us) server_negotiation.cc:407] Connection header received
0806 03:56:08.402768 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.402769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.402779 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.402808 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.403118 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34574: BlockingRecv error: recv got EOF from 127.0.0.1:34574 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260806 03:56:08.407117 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.406378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34590 (local address 127.12.15.125:35519)
0806 03:56:08.406410 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.406412 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.406638 (+ 226us) server_negotiation.cc:407] Connection header received
0806 03:56:08.406728 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.406729 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.406739 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.406767 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.407075 (+ 308us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34590: BlockingRecv error: recv got EOF from 127.0.0.1:34590 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260806 03:56:08.413024 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.412283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34594 (local address 127.12.15.125:35519)
0806 03:56:08.412316 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.412317 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.412520 (+ 203us) server_negotiation.cc:407] Connection header received
0806 03:56:08.412628 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.412629 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.412638 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.412665 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.412982 (+ 317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34594: BlockingRecv error: recv got EOF from 127.0.0.1:34594 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260806 03:56:08.423071 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.422263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34598 (local address 127.12.15.125:35519)
0806 03:56:08.422299 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.422301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.422531 (+ 230us) server_negotiation.cc:407] Connection header received
0806 03:56:08.422645 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.422646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.422658 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.422687 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.423022 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34598: BlockingRecv error: recv got EOF from 127.0.0.1:34598 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260806 03:56:08.441651 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.440678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34614 (local address 127.12.15.125:35519)
0806 03:56:08.440725 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.440727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.441006 (+ 279us) server_negotiation.cc:407] Connection header received
0806 03:56:08.441120 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.441122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.441135 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.441166 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.441587 (+ 421us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34614: BlockingRecv error: recv got EOF from 127.0.0.1:34614 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260806 03:56:08.475907 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.475128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34616 (local address 127.12.15.125:35519)
0806 03:56:08.475186 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.475188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.475362 (+ 174us) server_negotiation.cc:407] Connection header received
0806 03:56:08.475400 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.475401 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.475416 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.475479 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.475841 (+ 362us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34616: BlockingRecv error: recv got EOF from 127.0.0.1:34616 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
W20260806 03:56:08.541622 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.540998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34630 (local address 127.12.15.125:35519)
0806 03:56:08.541063 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.541065 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.541158 (+ 93us) server_negotiation.cc:407] Connection header received
0806 03:56:08.541211 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.541212 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.541232 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.541294 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.541570 (+ 276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34630: BlockingRecv error: recv got EOF from 127.0.0.1:34630 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260806 03:56:08.671604 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.670870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34642 (local address 127.12.15.125:35519)
0806 03:56:08.670947 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.670950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.671038 (+ 88us) server_negotiation.cc:407] Connection header received
0806 03:56:08.671124 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.671126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.671137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.671190 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.671535 (+ 345us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34642: BlockingRecv error: recv got EOF from 127.0.0.1:34642 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260806 03:56:08.929642 15010 negotiation.cc:337] Failed RPC negotiation. Trace:
0806 03:56:08.928938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.12.15.125:35519)
0806 03:56:08.928992 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:08.928993 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:08.929026 (+ 33us) server_negotiation.cc:407] Connection header received
0806 03:56:08.929093 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:08.929094 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:08.929103 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:08.929170 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0806 03:56:08.929567 (+ 397us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34648: BlockingRecv error: recv got EOF from 127.0.0.1:34648 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
I20260806 03:56:09.289911 15159 catalog_manager.cc:1259] Loaded cluster ID: b7b0db7d190c4b48a19b11df03faaeb5
I20260806 03:56:09.289985 15159 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5: loading cluster ID for follower catalog manager: success
I20260806 03:56:09.290833 15159 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5: acquiring CA information for follower catalog manager: success
I20260806 03:56:09.291145 15159 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b0ac14306acd45aeafd79b6c13f91eb5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:56:09.366989 15241 catalog_manager.cc:1259] Loaded cluster ID: b7b0db7d190c4b48a19b11df03faaeb5
I20260806 03:56:09.367085 15241 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a: loading cluster ID for follower catalog manager: success
I20260806 03:56:09.367954 15241 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a: acquiring CA information for follower catalog manager: success
I20260806 03:56:09.368278 15241 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 64467c904dc34afc936202ac2cc90a0a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260806 03:56:09.445464 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:09.442969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34658 (local address 127.12.15.125:35519)
0806 03:56:09.443082 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:09.443083 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:09.443094 (+ 11us) server_negotiation.cc:407] Connection header received
0806 03:56:09.443157 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:09.443158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:09.443214 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:09.443288 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:09.443499 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.444006 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.444761 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.444944 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.444988 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:09.445105 (+ 117us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:09.445248 (+ 143us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:09.445307 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:09.445373 (+ 66us) server_negotiation.cc:299] Negotiation successful
0806 03:56:09.445391 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":53,"threads_started":1}
I20260806 03:56:09.449481 15369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:09.447136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56642 (local address 127.12.15.124:46253)
0806 03:56:09.447235 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:09.447237 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:09.447245 (+ 8us) server_negotiation.cc:407] Connection header received
0806 03:56:09.447282 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:09.447283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:09.447320 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:09.447388 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:09.447472 (+ 84us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.447953 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.448755 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.448951 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.448999 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:09.449114 (+ 115us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:09.449225 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:09.449280 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:09.449372 (+ 92us) server_negotiation.cc:299] Negotiation successful
0806 03:56:09.449391 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80,"thread_start_us":50,"threads_started":1}
I20260806 03:56:09.453909 15360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:09.451298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.12.15.125:35519)
0806 03:56:09.451339 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:09.451341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:09.451522 (+ 181us) server_negotiation.cc:407] Connection header received
0806 03:56:09.451600 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:09.451601 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:09.451612 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:09.451658 (+ 46us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:09.451780 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.452232 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.453192 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.453428 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.453502 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:09.453620 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:09.453658 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:09.453701 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:09.453802 (+ 101us) server_negotiation.cc:299] Negotiation successful
0806 03:56:09.453828 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260806 03:56:09.453955 15369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:09.451152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56644 (local address 127.12.15.124:46253)
0806 03:56:09.451190 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:09.451192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:09.451383 (+ 191us) server_negotiation.cc:407] Connection header received
0806 03:56:09.451448 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:09.451449 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:09.451458 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:09.451515 (+ 57us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:09.451690 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.452408 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.453230 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.453509 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.453560 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:09.453685 (+ 125us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:09.453715 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:09.453764 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:09.453853 (+ 89us) server_negotiation.cc:299] Negotiation successful
0806 03:56:09.453880 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260806 03:56:09.454496 15380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0806 03:56:09.451230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43070 (local address 127.12.15.126:40467)
0806 03:56:09.451346 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0806 03:56:09.451348 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0806 03:56:09.451357 (+ 9us) server_negotiation.cc:407] Connection header received
0806 03:56:09.451407 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0806 03:56:09.451408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0806 03:56:09.451462 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0806 03:56:09.451538 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0806 03:56:09.451697 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.452450 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.453450 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0806 03:56:09.453733 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0806 03:56:09.453787 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0806 03:56:09.454152 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0806 03:56:09.454266 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0806 03:56:09.454327 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0806 03:56:09.454386 (+ 59us) server_negotiation.cc:299] Negotiation successful
0806 03:56:09.454413 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":57,"threads_started":1}
I20260806 03:56:09.455911 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14791
I20260806 03:56:09.470975 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 14905
I20260806 03:56:09.480947 12349 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskGVWHtl/build/release/bin/kudu with pid 15016
2026-08-06T03:56:09Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2902 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5842 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (274157 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST